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on  expected  future  performance.  In  the  first  phase.  Project  A,  a  large  and  versatile 
data  base,  was  collected  from  a  representative  sample  of  Military  Occupational 
Specialties  (MOS)  and  used  to  (a)  validate  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)  and  (b)  develop  and  validate  new  predictor  and  criterion  measures 
representing  the  entire  domain  of  potential  measures.  Building  on  this  foundation, 
Career  Force  research  will  finish  developing  the  selection/classification  system  and 
evaluate  its  effectiveness,  with  emphasis  on  assessing  second-tour  performance.  This 
second  year  of  the  project  emphasized  building  the  data  file  for  the  Longitudinal 
Validation  cohort,  developing  the  basic  scores  for  tlie  final  versions  of  the 
Experimental  Predictor  Battery  and  the  first-tour  performance  measures,  and  carrying 
out  the  ba.sic  longitudinal  validation. 


14  SUBJEa  terms 

Career  force 
Criterion  measures 
I'redictor  nieasures 


IT  security  CLASSIFICATION 
OF  REPORT 

•  •  w  •  -  i.  *  i  C  sj 


Nsrj  7>iU-0*.  280  SS00 


Longitudinal  validation 
Personnel  rla.s.si  f  ication 
Personnel  selection 


Project  A 
Second-tour 
performance 


IS.  NUMBER  OF  PAGES 

368 


16.  PRICE  CODE 


IB  security  aASSIFICATlON  119.  security  CLASSIFICATION  I  20  LC.  . .  ATlON  OF  ABSTRACT 


OF  This  page 

Unc  J  a.ssl  f  ied 


OF  abstract 

Unclassi f 1 ed 


Uni  1  mi  t fid 


St»nd*rd  fo^m  298  (Rpv  2  89) 

nrv'-ora  t>»  ‘NLi  Lid 


FOREWORD 


This  document  is  a  description  of  the  research  activities  conducted 
during  the  second  year  of  the  project  Building  the  Career  Force.  This  project 
is  the  second  phase  of  a  research  program  of  unprecedented  scope  and  depth  to 
provide  the  basis  for  improving  the  Army's  selection  and  classification 
procedures,  as  well  as  improving  reenlistment  and  promotion  decisions  for 
soldiers  up  to  the  level  of  sergeant.  The  thrust  for  this  program  came  from 
the  practical,  professional,  and  legal  need  to  validate  the  Armed  Services 
Vocational  Aptitude  Battery  (ASVAB--the  current  U.S.  military  selection/ 
classification  test  battery)  and  other  selection  variables  as  predictors  of 
training  and  performance.  The  authorization  for  the  program  was  provided  in  a 
letter.  Deputy  Chief  of  Staff  for  Operations,  "Army  Research  Project  to 
Validate  the  Predictive  Value  of  tiie  Armed  Services  Vocational  Aptitude 
Battery,"  effective  19  November  1980,  and  a  Memorandum,  Assistant  Secretary  of 
Defense,  Manpower  Reserve  Affairs  and  Logistics  (MRA&L) ,  "Enlistment 
Standards,"  effective  11  September  1980. 

The  research  program  began  in  1982  with  an  effort  known  as  Project  A. 
Project  A  not  only  validated  the  ASVAB  against  job  performance;  it  further 
linked  indicators  of  temperament  (achievement,  discipline,  stress  tolerance), 
psychomotor  ability  (e.g.,  eye-hand  coordination),  and  spatial  ability  to  job 
performance.  Project  A  developed  new  tools  for  a  variety  of  personnel 
decisions.  Before  these  tools  can  be  optimally  used,  however,  two  critical 
questions  need  to  be  answered:  (1)  What  combinations  of  aptitude, 
temperament,  psychomotor  ability,  and  spatial  ability,  measured  at  or  before 
entry  into  the  Army,  best  predict  later  performc.nce  in  individual  military 
occupational  specialties  (MOS)?  (2)  Which  indicators  of  first-tour 
performance  best  predict  performance  in  the  second  tour?  These  questions  will 
be  answered  in  Building  the  Career  Force. 

The  second-year  Building  the  Career  Force  activities  described  herein 
continued  analyses  of  the  combined  set  of  initial  entry  predictor  measures 
developed  for  selection  and  classification  purposes  and  end-of-training  and 
first-tour  job  performance  measures  to  be  linked  to  these  predictor  measures. 
Preliminary  second-tour  measures  were  refined  and  administration  was  begun  to 
a  sample  already  tested  on  initial  entry,  end-of-training,  and  first-tour 
measures.  After  administration  of  the  second-tour  measures  is  completed  in 
1992,  it  will  be  possible  to  exainim.  1 '^ngi tudi nal  linkages  across  the  full  set 
of  measures,  from  initial  entry  to  second  tour.  This  will  provide  an 
information  base  for  setting  selection,  classification,  reenlistment,  and 
promotion  policies  unrivaled  anywhere. 


Active  sponsorship  of  this  effort  has  been  provided  by  the  Director  of 
Military  Personnel  Management  (DMPM).  The  OMPM  has  been  periodically  briefed 
on  the  activities  described  herein  and  has  personally  taken  part  in  the 
execution  of  this  project  in  an  extremely  effective  manner.  To  ensure  that 
Building  the  Career  Force  research  achieves  its  full  scientific  potential,  an 
advisory  group  composed  of  experts  in  personnel  measurement,  selection,  and 
classification  was  established  to  provide  continuing  guidance  on  technical 
aspects  of  the  research.  Members  of  this  Scientific  Advisory  Group  include 
Philip  Bobko,  Lloyd  Bond,  Milton  Hakel  (Chair),  Lloyd  Humphreys,  Lawrence 
Johnson,  Robert  Linn,  Mary  Tenopyr,  and  Jay  Uhlaner. 


EDITORS'  PREFACE 


This  is  the  second  annual  report  for  work  completed  as  part  of  the 
Building  the  Career  Force  project.  It  also  constitutes  the  primary  technical 
report  of  the  work  completed  on  several  of  the  project's  principal  tasks. 
Consequently,  it  is  a  "stand  alone"  document  for  Fiscal  Year  1991  and  does  not 
refer  the  reader  to  more  detailed  descriptions  in  supplementary  reports.  The 
Career  Force  project  extends  the  major  work  on  selection  and  classification  of 
Army  enlisted  personnel  that  was  completed  as  part  of  Project  A. 

The  Career  Force  project  includes  (1)  a  replication  and  extension  of  the 
Experimental  Battery  validities  for  the  selection  and  classification  of  first- 
tour  enlisted  personnel;  (2)  validation  of  the  Experimental  Battery  against 
end-of-training  performance;  (3)  validation  of  training  performance  as  a 
predictor  of  first-tour  job  performance;  (4)  measurement  of  second-tour 
performance;  (5)  validation  of  the  Armed  Services  Vocational  Aptitude  Battery 
(ASVAB) ,  the  Experimental  Battery.  Advanced  Individual  Training  (AIT) 
performance,  and  first-tour  performance  as  predictors  of  second-tour 
performance;  and  (6)  identification  of  the  optimal  predictor  battery  for 
selection  and  classification,  given  certain  specific  sets  of  goals  and 
constrai nts . 

The  annual  report  for  year  one  described  the  results  of  a  series  of 
analyses  directed  at  basic  score  development  for  (1)  the  Experimental 
Predictor  Battery,  (2)  the  End-of-Training  performance  measures,  and  (3)  the 
second-tour  job  performance  measures  that  were  administered  to  the  second-tour 
Concurrent  Validation  sample  (CVII).  The  performance  data  from  this  initial 
sample  of  second-tour  junior  noncommissioned  officers  were  also  used  to 
develop  a  latent  structure  model  of  second-tour  performance.  The  model 
hypothesizes  six  basic  components  for  NCO  performance. 

This  Research  Note  deals  with  the  analysis  of  performance  data  from  the 
Longitudinal  Validation  I  (LVI)  sample,  which  is  a  sample  of  approximately 
10,000  first-tou’"  incumbents  who  entered  the  Army  during  1986/87.  It  is  the 
second  of  the  two  major  cohorts  of  enlisted  personnel  thct  make  up  the  total 
Project  A/Career  Force  project  data  base.  The  criterio;i  score  development, 
data  editing,  and  performance  modeling  analyses  are  each  described  in  turn. 
After  the  performance  factor  scores  have  been  finalized,  the  remainder  of  the 
report  deals  with  the  basic  Longitudinal  Validation  of  the  ASVAB  and  the 
Project  A  Experimental  Predictor  Battery  against  (1)  training  performance,  (2) 
first-tour  job  performance,  and  (3)  second-tour  job  performance  (i.e.,  the 
second-tour  performance  factor  scores  developed  during  year  one). 


V 


The  basic  building  blocks  are  now  in  place  for  (1)  development  of 
"optimal"  prediction  equations,  given  constraints;  (2)  estimation  of  potential 
differential  prediction/classific.’tion  validity;  and  (3)  analysis  of  the 
tradeoffs  among  alternative  selection  and  classification  goals.  The  results 
of  these  analyses  will  be  the  topics  of  subsequent  reports. 

As  was  the  case  for  year  one,  the  writing  of  this  report  was  very  much  a 
collaborative  effort  by  a  lot  of  people.  The  primary  authors  for  each  chapter 
are  indicated  in  the  Table  of  Contents  and  also  on  the  first  page  of  each 
chapter.  The  editors,  and  the  management,  are  deeply  appreciative  of  their 
coritr  ibutions. 
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BUILDING  AND  RETAINING  THE  CAREER  FORCE;  NEW  PROCEDURES  FOR  ACCESSING  AND 
ASSIGNING  ARMY  ENLISTED  PERSONNEL 
ANNUAL  REPORT.  1991  FISCAL  YEAR 


EXECUTIVE  SUMMARY 


Reqiii  rement : 

The  Building  the  Career  Force  project  is  the  second  phase  of  a 
comprehensive,  long-term  research  program,  sponsored  by  the  Deputy  Chief  of 
Staff  for  Personnel,  to  provide  the  basis  for  improving  the  selection  and 
assignment  of  Army  enlisted  personnel.  In  the  first  phase,  known  as  Project 
A,  existing  selection  measures  were  validated  against  both  existing  and  newly 
developed  performance  criteria,  and  new  predictive  measures  were  developed  to 
aid  in  assignment  and  promotion  decisions.  The  Career  Force  project  extends 
the  research  on  measuring  second-tour  job  performance  and  is  examining  how 
selection  and  classification  tests  administered  before  a  soldier's  first 
enlistment  can,  with  measures  of  performance  during  that  enlistment,  predict 
performance  potential  for  second-tour  duty. 


Procedure: 

In  Task  1,  measures  developed  in  Project  A  to  assess  the  performance  of 
second-tour  soldiers  have  been  revised  and  are  being  tested  with  the 
Longitudinal  Validation  sample  first  tested  in  Project  A.  (These  second-tour 
soldiers  have  been  in  the  Army  from  41  to  63  months.)  The  results  from  these 
tests  are  being  used  to  complete  the  predictive  validation  of  tiie  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB)  and  the  Project  A  Experimental 
Predictor  Battery  against  training  success  measures  and  first-tour  job 
performance  tests,  and  of  all  of  these  measures  against  the  criteria  for 
successful  second-tour  performance. 

The  Task  2  staff  has  established,  and  is  managing,  an  integrated 
research  data  base  (IRDB),  processing  Project  A  and  Career  Force  data  and 
merging  files  with  unrelated  military  data. 
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Task  3  covers  all  analyses  to  be  performed  under  this  project  to  develop 
the  analytic  framework  needed  to  evaluate  equations  for  predicting  training 
performance,  first-tour  performance  and  attrition,  reenl 1 stment,  and  second- 
tour  performance. 


Findi ngs; 

The  pattern  of  results  from  the  basic  analyses  of  Longitudinal 
Validation  tests  was  highly  consistent  with  the  results  from  the  earlier 
Concurrent  Validation  tests  for  both  predictor  and  performance  scores.  For 
example,  the  model  of  first-tour  performance  that  best  fit  the  Longitudinal 
Validation  data  was  the  model  generated  from  the  Concurrent  Validation 
testing. 

Similarly,  the  models  for  training  performance  and  for  second-tour 
performance  that  have  so  far  been  developed  from  the  Longitudinal  Validation 
data  have  strongly  confirmed  the  earlier  findings.  The  description  of  the 
latent  structure  of  performance  as  Individuals  move  from  training  through 
their  first  tour  and  Into  their  second  tour  is  highly  consistent. 

The  ASVAB  was  again  shown  to  be  a  very  good  predictor  of  the  "will  do" 
component  of  performance  as  well  as  a  reasonably  good  predictor  of  the  effort 
and  leadership  aspect  of  performance. 


Utilization  of  Findings: 

The  findings  from  the  basic  validation  stage  will  be  used  to  consider  a 
variety  of  issues  inherent  in  optimal  prediction  of  performance.  The  long¬ 
term  results  from  these  developmental  and  validation  processes  will  be  applied 
in  an  Improved  system  for  selecting  and  assigning  Army  manpower  in  a  changing 
military  environment. 
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BUILDING  AND  RETAINING  THE  CAREER  FORCE: 

NEW  PROCEDURES  FOR  ACCESSING  AND  ASSIGNING  ARMY  ENLISTED  PERSONNEL 

ANNUAL  REPORT.  1991  FISCAL  YEAR 

Chapter  1 
INTRODUCTION 

John  P.  Campbell  and  James  H.  Harris 


This  report  is  a  summary  of  the  major  activities  undertaken  during  the 
second  year  of  a  Department  of  the  Army  research  project  entitled  Building  and 
Retaining  the  Career  Force.  The  report  covers  the  period  from  1  October  1990 
through  30  September  1991.  The  research  reported  was  conducted  by  a 
consortium  comprised  of  Human  Resources  Research  Organization  (HumRRO), 
American  Institutes  for  Research  (AIR),  and  Pe«'sonnel  Decisions  Research 
Institute,  Incorporated  (PORI,  Inc.),  and  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  ^ARI). 

The  research  effort  is  the  second  phase  of  a  two-phas'’  program  to 
develop  a  selection  and  classification  system  for  enlisted  personnel,  based  on 
expected  future  performance.  Phase  One  was  Project  A  (Campbell  &  Zook,  1990, 
1991).  Its  goals  were  to  validate  the  Armed  Services  Vocational  Aptitude 
Battery  (ASVAB)  by  collecting  data  from  a  representative  sample  of  Military 
Occupational  Specialties  (MOS),  and  to  build  a  large  and  versatile  data  base 
by  developing  and  validating  new  predictors  and  criterion  measures  that 
represented  the  entire  domain  of  potential  measures.  The  goals  of  Building 
the  Career  Force  are  to  determine  the  longitudinal  relationship  between  the 
new  predictors  and  first-tour  performance,  to  finalize  and  administer  the 
measures  of  second-tour  job  performance,  and  to  examine  how  selection  and 
classification  tests  administered  before  a  soldier's  first  enlistment,  in 
conjunction  with  performance  during  that  soldier's  first  enlistment,  predict 
performance  in  a  second  enlistment. 

The  remainder  of  this  chapter  describes  the  objectives  and  organization 
of  the  project,  summarizes  the  work  completed  during  the  first  15  months,  and 
outlines  the  content  to  be  included  in  this  second  annual  report. 


BUILDING  THE  CAREER  FORCE;  OBJECTIVES  AND  PROJECT  DESIGN 

The  Project  A  data  base,  the  predictor  and  criterion  measures  the 
project  developed,  the  working  models  it  provided,  and  its  basic  analytic 
work  have  provided  a  valuable  foundation  for  the  further  production  of 
scientific  findings  and  operational  products,  and  for  the  subsequent 
investigation  of  reenlistment  decisions,  noncommissioned  officer  (NCO)  job 
performance,  NCO  promotion  decisions,  and  the  identification  of  NCO  potential. 

The  work  encompassed  by  the  Career  Force  project  is  intended  to 
accomplish  several  general  goals  relevant  to  building  and  retaining  the  career 
force: 


(1)  Build  the  final  pieces  required  for  a  complete  selection/ 
classification  decision-making  system. 
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(2)  Provide  the  analytic  procedures  and  data  required  to  maximize  the 
system's  performance  and  evaluate  its  effectiveness. 

(3)  Build  the  foundation  for  its  implementation. 

The  principal  focus  is  on  the  greatest  possible  gains  in  overall 
individual  performance,  for  both  "can  do"  and  "will  do"  components  of  perfo 
ance,  that  can  be  obtained  from  enhancing  the  selection/classification  systew 
for  first-  and  second-tour  enlisted  personnel.  Maximizing  the  benefit  from  a 
more  effective  match  of  people  and  jobs  has  always  been  a  goal  of  the  Army 
personnel  system.  Given  the  population  demographics  for  the  United  States 
during  the  coming  decade,  this  goal  becomes  even  more  crucial.  It  is 
incumbent  on  virtually  every  organization  to  go  as  far  as  the  state-of-the-art 
will  al low. 

This  means  that  the  information  that  is  used  to  make  personnel  decisions 
must  yield  the  maximum  gain  in  terms  of  accuracy  and  fairness  of  predictions. 
It  means  that  the  models  and  procedures  used  to  execute  selection  and 
classification  decisions  must  both  serve  the  goals  of  the  organization  and 
maximize  the  aggregate  benefits  that  can  be  obtained  from  using  the  selection/ 
classification  measures  (e.g.,  new  computerized  tests).  It  means  that  the 
implementation  of  the  system,  or  any  part  of  it,  must  serve  the  needs  of  the 
users  and  also  maintain  fidelity  with  the  goals  on  which  the  system  is  based. 


Specific  Research  Objectives 

The  specific  scientific  objectives  of  Building  the  Career  Force  are  to 

(1)  Develop  a  complete  array  of  valid  and  reliable  measures  of  second- 
tour  performance  as  an  Army  NCO,  using  the  Project  A  prototypes  as 
a  starting  point. 

(2)  Carry  out  a  complete  incremental  predictive  validation  of  (a)  the 
ASVAB  and  the  Project  A  Experimental  Battery  of  predictors,  (b) 
measures  of  training  success,  and  (c)  the  full  array  of  first-tour 
performance  criteria  developed  as  part  of  Project  A.  The  criteria 
against  which  these  three  sets  of  predictors  will  be  validated, 
both  individually  and  incrementally  for  each  major  criterion 
component,  are  the  second-tour  job  performance  measures. 

(3)  Develop  a  model  of  second-tour  NCO  performance  that  parallels  the 
first-tour  performance  model  from  Project  A  and  that  identifies 
the  major  components  of  second-tour  performance,  provides 
information  on  their  construct  validity,  and  establishes  how  the 
major  components  of  performance  should  be  combined  for  specific 
prediction  or  interpretation  purposes. 

(4)  Develop  the  analytic  framework  needed  to  evaluate  the  optimal 
prediction  equations  for  predicting  (a)  training  performance; 

(b)  first-tour  performance;  (c)  first-tour  attrition  and  the 
reenlistment  decision;  and  (u)  second-tour  performance,  under  the 
conditions  when  testing  time  is  limited  to  a  specified  amount  and 
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when  there  must  be  a  tradeoff  among  alternative  selection/ 
classification  goals  (e.g.,  maximizing  aggregate  performance  vs. 
minimizinq  discipline  and  low-motivation  problems  vs.  minimizing 
attrition) . 

(5)  Design  and  develop  a  fully  functional  and  user-friendly  research 
data  base  that  includes  all  relevant  personnel  data  on  1981/82, 
1983/84,  and  1986/87  accessions,  including  all  Project  A  and 
Career  Force  Project  data  and  all  relevant  Enlisted  Master  Files 
(EMF),  Accession  File,  and  Army  Training  Requirements  and 
Resources  System  (ATRRS)  available  data. 


Project  Organization 

To  reflect  the  requirements  of  the  research,  the  project  is  organized  as 
shown  in  Figure  1.1.  Management  of  the  total  project  is  the  responsibility  of 
the  Project  Director.  The  overall  design,  execution,  and  evaluation  of  the 
substantive  tasks  are  the  responsibility  of  the  Principal  Scientist. 

Oversight  and  scientific  participation  are  provided  by  the  Army  Research 
Institute.  Guidance  is  provided  by  the  General  Officers  Steering  Committee 
and  the  Scientific  Advisory  Group. 

A  brief  summary  of  the  work  encompassed  by  the  three  substantive 
technical  tasks  follows: 

Task  1  is  to  revise  the  measures  developed  in  Project  A  to  measure 
second-tour  soldier  performance.  The  second-tour  performance  measures  were 
revised  and  were  administered  to  the  Project  A  Longitudinal  Validation  (LV) 
sample,  beginning  in  June  1991.  At  that  time,  the  soldiers  in  the  sample  were 
in  their  second  tour,  and  had  been  in  the  Army  anywhere  from  to  63  months. 
Once  these  measures  have  been  administered,  and  the  data  analyzed  (under  Task 
3),  it  will  be  possiole  to  complete  the  incremental  predictive  validation  of 
the  ASVA6  and  the  Project  A  Experimental  Battery,  the  measures  of  training 
success,  and  the  full  array  of  first-tour  performance  measures  developed  in 
Project  A,  against  the  second-tour  criterion  measures. 

Task  2  has  a  single  purpose--to  establish,  manage,  and  safeguard  an 
integrated  research  data  base  (IRD8)  on  the  National  Institutes  of  Health  IBM 
computer  system.  As  part  of  the  establishment  of  the  IRDB,  Task  2  will 
integrate  the  Project  A  longitudinal  research  data  base,  extract  and  merge 
data  from  other  military  data  bases,  process  data  collected  by  Project  A  and 
tnis  project,  and  create  workfiles  for  analyses. 

Task  3  is  responsible  for  all  analyses  performed  under  this  project. 

The  task  is  organized  around  the  five  major  data  sets  to  be  analyzed.  The 
data  sets  are  the  Longitudinal  Validation  predictor  data  (LV),  the 
Longitudinal  Validation  end-of-training  (EOT)  data,  the  Longitudinal 
Validation  first-tour  data  (LVI),  the  Concurrent  Validation  second-tour  data 
(evil),  and  the  Longitudinal  Validation  second-tour  data  (LVII).  At  the  end 
of  the  project,  Task  3  will  have  developed  the  analytic  framework  necessary  to 
evaluate  optimal  prediction  equations  to  predict  training  performance,  first- 
tour  performance  and  attrition,  reenlistment,  and  second-tour  performance. 
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gure  l.l.  Building  the  Larcer  force:  I'roject  nianaQ.  .lent  structure. 


Project  Design 


As  will  be  explained  in  a  later  section  of  this  chapter,  the  remaining 
chapters  of  this  report  all  deal  with  analyses  of  data  obtained  at  various 
points  in  the  total  project  design.  To  set  the  stage  for  these  discussions, 
as  well  uS  for  the  summary  of  work  done  during  year  one,  the  basic  project 
design  is  sur-narized  below. 

The  Research  Sample 

In  general,  the  combined  design  for  Project  A/Career  Force  encompasses 
two  major  cohorts  of  soldiers  (new  accessions  for  1983/84  and  for  1986/87), 
each  of  which  were  followed  into  their  second  tour  of  duty  and  which 
collectively  have  produced  six  major  research  samples  For  each  research 
sample  there  is  a  battery  of  predictor  measures  and  an  array  of  performance 
measures.  For  each  of  the  six  samples  the  predictor  battery  is  composed  of 
the  ASVAB  and  either  the  Trial  Battery  or  the  Experimental  Battery  version  of 
the  new  tests  developed  in  Project  A  (see  Campbell  &  Zook,  1991).  There  were 
three  distinct  arrays  of  performance  measures  corresponding  to  the  need  to 
assess  (a)  training  performance,  (b)  first-tour  job  performance,  and  (c) 
second-tour  job  performance. 

In  each  sample  the  individuals  to  be  assessed  were  selected  from  tw\, 
predetermined  sets  of  MOS  --  Batch  A  and  Batch  Z.  They  are  listed  in  Figure 
1.2.  The  two  groups  differed  in  that  tests  administered  to  Batch  A  MOS 
included  MOS-specific  rating  scales  and  hands-on  and  job  knowledge  tests, 
whereas  the  only  MOS-specific  measure  administered  to  the  Batch  Z  MOS  was  an 
end-of-training  test. 


Baich  A 

Batch  Z 

MOS 

MOS 

11B  Infantryman 

12B  Combat  Engineer 

13B  Cannon  Crewman 

16S  MANPADS  Crewman 

1 9E  M60  Armor  Crewman 

27E  Tow/Dragon  Repairer 

19K  Ml  Armor  Crewman 

29E  Comm. -Electronics  Radio  Repairer 

31 C  Single  Channel  Radio  Operator 

51 B  Carpentry/Masonry  Specialist 

63B  Light-Wheel  Vehicle  Mechanic 

54B  NBC  Specialist 

71 L  Administrative  Spec.alist 

55B  Ammunition  Specialist 

88M  Motor  T ransport  Operator 

67N  Utility  Helicopter  Repairer 

91 A  Medical  Specialist 

76Y  Unit  Supply  Specialist 

95B  Military  Police 

94B  Food  Service  Specialist 

96B  Intelligence  Analyst 

Figure  1.2.  Project  A/Career  Force  Military  Occupational  Specialties  imuS, 
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The  MOS  in  the  two  groups  were  carefully  sampled  to  represent  the 
iation  in  job  content  in  the  Army  occupational  structure.  In  addition, 
were  selected  so  as  to  overrepresent  both  the  combat  specialties  and 
e  MOS  with  the  larger  proportions  of  women  and  minority  groups.  The  MOS 
ction  procedure  has  been  described  in  detail  in  previous  Project  A  reports 
(•  3 . ,  Campbel  i ,  1987 ) . 

A  glossary  of  terms  for  the  samples  and  for  the  different  measurement 
L:  teries  is  given  in  Figure  1.3.  The  six  major  samples,  their  approximate 
S'  and  the  predictor  and/or  performance  batteries  that  were  administered  to 
e .  n  are  shown  in  Figure  1.4. 


Glossary  of  Terms 

1  CVI  Sample  (CVI) 

1 

1 

Soidiers  who  entered  the  Army  between  1  Jul  83  ■  30  Jun  84  and  were  in 

1 985  Project  A  Concurrent  Validation.  They  were  administered  ’he  Trial 
Pfodictof  Battery  and  the  first-tour  job  performance  measures.  j 

evil  Sample  (CVII) 

i 

1 

Soldiers  who  entered  the  Army  between  1  Jul  83  •  30  Jun  84  and  were  in 
the  1 985  Protect  A  Corxxirrent  Validation  (CVI)  and  the  1988 

Second-tour  Concurre'  it  Validation  (CVII).  They  were  administered  the 
second-tour  job  performance  measures  and  were  re-admimstered  the 

ABLE. 

'  LV  Sample  (LV) 

! 

Soldiers  m  the  Longitudinal  Validation  sample  who  entered  the  Anny 
between  20  Aug  86  -  30  Nov  87  aod  were  administered  the  Expenmental 
Predictor  Battery  and  End-o(-Training  measures. 

LV  Training 

Sample  (LVT) 

Soldiers  in  the  Longitudinal  Validation  sample  who  finished  AIT  and  who 
were  administered  the  End-oi-Trainmg  measures. 

1  LVI  Sample  (LVI) 

Soldiers  who  entered  the  Army  between  20  Aug  86  •  30  Nov  87  and 
were  m  the  LV  Sample  aiid  the  1988  First-  Tour  Longitudinal  Validaiion 

Sample.  They  were  administered  the  first-tour  )ob  performance 
measures. 

LVIl  Sample  (LVIl) 

Soldiers  who  entered  the  Amy  between  20  Auo  86  -  30  Nov  87  and 
were  in  the  LV  Sample  and  the  LVI  Sample  aofl  who  will  be  m  the  1991 
LongitudinaJ  Validation  (LVIl).  They  were  administered  the  second-tour 
job  performance  measures.  | 

i  gure  1.3.  Glossary  of  terms  for  Project  A/Career  Force  research  samples. 
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81/82 

Oaia 

Arutyw 


Prgdicnr 

eanary 

and 

FiiK-Touf 

P«rlormanc« 

Measumi 

N.8S00 


83/84 

Concurrsn 

Samp<4 

S«cood-lo>« 

Padormacna 

Uaaturta 

N.1000 

cvg 


86/87  Longitudmai  S«np« 


-ot- 


£ip*nm«rtal 

Pnaaor 

Bantry 

N.iVOOO 


86/87  1 

"5 

8&B7 

Usngnudtnal 

■V 

UongiudinU 

SampK 

Sanoic 

FiroTouf 

1  ^ 

o«coo(J.Toor 

Pirtofmane* 

Padominca 

UoasiTM 

Mtasum 

N.  10.000 

1 

N.-i.*0O 

LVJ 

4 

LVIl 

Figure  1.4.  Career  Force  research  flow  and  samples. 


Procedure 

The  data  collection  procedures  for  each  sample  have  been  described  in 
detail  in  previous  reports  (e.g.,  see  Campbell  &  Zook,  1990).  Each  data 
collection  involved  cn-site  administration  by  a  trained  data  collect'on  team 
headed  by  a  team  leader  from  the  contractor  staff  who  worked  closely  with  a 
designated  Army  po i nt-of-contact  (POC)  at  ttie  site.  A  brief  characterization 
of  each  of  the  six  samples  in  terms  of  the  timing,  location,  and  duration  (per 
soldier)  of  the  data  collection  is  given  below. 

The  Concurrent  Validation  (CVI)  sample.  The  data  were  collected  at  13 
posts  in  the  continental  United  States  and  at  two  locations  in  Europe.  Each 
individual  was  assessed  for  1  1/2  days  on  the  project-developed  first-tour  job 
performance  measures  and  for  1/2  day  on  the  new  predictor  measures  (the  Trial 
Battery).  The  individuals  in  the  sample  had  been  in  the  Army  for  18-24 
months.  Data  analysis  has  been  completed  for  this  sample. 


The  Longitudinal  Validation  (LV)  Sample.  All  individuals  were  assessed 
on  the  4-hour  Experimental  Predictor  Battery  within  2  days  of  first  arriving 
at  their  assigned  Reception  Battalion  where  they  would  unoergo  Basic/Advanced 
Individual  training.  Data  were  collected  over  a  14-month  period  at  eight 
Reception  Battalions  by  a  permanent,  on-site  data  collection  team. 

The  Longitudinal  Validation  Eno-of- iraining  Sample  (LVT).  The  E./J-o'- 
Training  (EOT)  performance  measures  were  administered  to  those  individuals  in 
the  LV  sample  w  '  completed  Advanced  Individual  Training  (AIT),  which  could 
take  from  2  m.  ns  to  6  months,  depending  on  the  MOS.  The  training 
performance  measures  consisted  of  an  MOS-specific  training  achievement  test 
and  a  series  of  rating  scales  completed  by  peers  and  drill  instructors.  Data 
collection  took  place  during  the  last  three  days  of  AIT. 

The  Longitudinal  Performance  Measurement  (LVI)  Sample.  The  individuals 
in  the  86/87  cohort  who  were  measured  with  the  Experimental  Predictor  Battery, 
completed  AIT,  and  remained  in  the  Army  were  assessed  with  the  full  array  of 
first-tour  job  performance  measures  when  they  were  between  18  and  24  months  of 
service.  Data  collections  were  conducted  at  13  posts  in  the  United  States  and 
two  locations  in  Europe.  The  administration  of  the  LVI  first-tour  criterion 
measures  took  one  day  per  soldier. 

The  Concurrent  Validation  Second-Tour  (CVIl)  Sample.  The  same  data 
collection  teams  that  administered  the  first-tour  performance  measures  to  the 
LVI  sample  also  administered  the  second-tour  performance  measures  at  the  same 
location  and  during  the  same  time  periods  to  a  sample  of  junior  NCOS  from  the 
33/84  cohort  in  their  second  tour  of  duty  (4-5  years  of  service).  Every 
attempt  was  made  to  include  second-tour  personnel  from  the  designated  MOS  who 
had  been  part  of  the  first-tour  Concurrent  Validation  sample  (CVI).  The  CVII 
data  collection  took  one  day  per  soldier. 

The  Longitudinal  Validation  Second-Tour  (LVII)  Sample.  The  personnel  in 
this  sample  are  members  of  the  86/87  cohort  from  the  designated  MOS  who  were 
part  of  the  LV  (predictors  and  training  performance  measures)  and  LVI  (first- 
tour  job  performance  measures)  samples  and  who  reenlisted  for  a  second  tour  of 
duty.  The  revised  second-tour  performance  measures  are  being  administered  at 
15  U.S.  posts,  two  locations  in  Europe,  and  two  locations  in  Korea.  The  LVII 
performance  assessment  takes  one  day  per  soldier. 

Current  Status 

The  LVII  data  collection  is  ongoing  and  will  be  completed  during  the 
summer  of  1992.  The  content  of  this  second-year  annual  report  is  based  on 
analyses  of  data  from  the  LV,  LVI,  and  CVII  samples. 


ADMINISTRATIVE  SUMMARY 

The  project  advisory  group  meetings  and  In-Progress  Reviews  (IPR)  during 
tne  first  two  years  of  the  project  are  summarized  in  this  section.  The 
Scientific  Advisory  Group  (SAG)  and  an  Action  Officers'  Working  Group  met  once 
each  during  this  period.  In  addition,  project  staff  members  and  sponsor 
representatives  held  two  IPRs  for  Tasks  2  and  3. 


Scientific  Advisory  Group 

The  purpose  of  the  SAG  meeting  was  to  present  the  results  of  the 
following  activities: 

•  Completion  of  the  data  entry  and  data  editing  necessary  to  produce 
appropriate  data  files  for  analyses  of  the  Longitudinal  Validation 
predictor  data  (LV  predictors)  collected  during  Project  A;  the 
longitudinal  sample  end-of-training  (EOT)  performance  data 
collected  during  Project  A;  and  the  second-tour  (NCO)  performance 
data  collected  on  che  83-84  cohort  as  part  of  the  Project  A 
Longitudinal  Validation.  This  is  the  Concurrent  Validation  II 
(evil )  sample. 

•  The  "basic  scores"  by  which  the  information  in  the  Experimental 
Predictor  Battery,  the  EOT  performance  measures,  and  the  CVII 
performance  measures  was  represented. 

•  The  modeling  of  the  latent  structure  of  the  covariance  matrices 
for  each  of  the  three  sets  of  basic  scores  measuring  performance. 

During  the  meeting,  little  time  was  spent  on  the  data  quality  analyses 
or  basic  item/scale  statistics.  The  major  part  of  the  attention  was  devoted 
to  discussions  on  the  rationale  for  the  basic  scores  in  each  domain  and  the 
subsequent  analysis  of  their  covariance  structures  and  whether  the  subsequent 
modeling  steps  constituted  a  useful  procedure. 

The  SAG  endorsed  the  project's  overall  objectives  but  expressed  caution 
that  immediate  Army  problems  and  priorities  not  be  overlooked.  They 
emphasized  that  specific  analysis  plans  should  be  linked,  where  possible,  to 
relevant  operational  issues;  however,  concern  for  operational  relevance  should 
not  be  a  basis  for  sacrificing  scientific  quality. 

With  respect  to  the  analyses  of  second-tour  measures,  there  was 
considerable  discussion  about  both  the  analytic  methods  used  and  the  results 
obtained.  The  SAG  recommended  that  special  attention  be  given  to  the  problem 
of  restricted  range  in  the  measures  of  second-tour  performance.  In 
particular,  they  were  interested  in  comparisons  between  those  who  "made  it" 
and  those  who  did  not.  They  also  recommended  separate  analyses  on  combat  and 
noncombat  jobs. 

The  SAG  has  a  strong  interest  in  additional  analyses  focused  on 
identifying  more  precisely  what  the  Situational  Judgment  Test  (SJT)  is 
measuring.  They  also  believe  that  SJT  has  a  high  potential  as  an  operational 
tool  and  that  such  an  implementation  should  be  considered. 

The  SAG  suggested  that  the  distinction  between  leadership  and  technical 
skill/effort  be  more  closely  examined  in  the  analyses  of  second-tour 
performance  dimensions.  The  ratings  method  variance  that  is  confounded  with 
many  of  the  reference  variables  for  the  leadership  construct  may  have  made  it 
difficult  for  a  unique  leadership  factor  to  emerge.  Some  further  analyses 
were  suggested  for  consideration.  First,  a  close  analysis  of  the  residuals 
was  thought  to  be  useful.  Second,  an  analysis  with  all  indicators  of 
leadership  removed  was  suggested.  Finally,  there  was  a  suggestion  that  a 
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small  sample  be  drawn  representing  the  distribution  of  combat  and  noncombat 
forces,  and  that  separate  analyses  be  run  on  this  sample. 

With  respect  to  end-of-training  analyses,  the  SAG  expressed  concern 
about  the  large  jmber  of  ratees  per  rater  in  some  instances.  They  advised 
further  consideration  of  whether  too  many  factors  were  identified  in  the 
analyses  of  the  end-of-training  data.  They  recommended  that  when  the  training 
measures  are  examined  as  predictors  of  future  performance,  multiple  regression 
be  conducted  first  with  one  general  end-of-training  factor,  with  the  other 
factors  added  one-by-one.  In  this  way,  the  incremental  value  of  each 
additional  factor  can  be  determined. 

Several  reconmiendations  were  made  regarding  analyses  of  the  predictor 
measures.  SAG  suggested  that  a  complete  theoretical  foundation  for  the 
analyses  concerning  modeling  of  predictor  measures  be  developed.  They  raised 
warnings.  First,  they  cautioned  project  staff  against  ignoring  factors  that 
may  seem  small;  careful  attention  should  be  given  to  extracting  all  relevant 
factors  for  the  ABLE  questionnaire  (Assessment  of  Background  and  Life 
Experiences),  for  ASVAB,  and  for  the  spatial  tests.  SAG  also  cautioned 
against  trying  too  vigorously  to  develop  interest  scales  that  balance  gender 
differences . 


Action  Officers'  Working  Group 

The  Action  Officers'  Working  Group  is  comprised  of  representatives  from 
the  major  commands  who  were  to  support  the  1991/92  second-tour  data  collection 
(LVII).  The  Group  was  established  by  ARI  to  reflect  the  need  to  involve  the 
elements  of  the  Army  most  directly  concerned  at  critical  points  throughout  the 
project--to  get  input  and  guidance  on  critical  issues  confronting  the  project 
so  that  these  issues  are  resolved  in  a  manner  which  best  serves  the  interests 
of  al 1  involved. 

The  specific  purpose  of  this  meeting  was  to  provide  the  Group  with  a 
general  understanding  of  the  project  and  its  goals  and  solicit  their  thoughts 
and  suggestions  on  (a)  the  general  focus  and  direction  of  the  project,  (b)  the 
manner  in  which  NCO  performance  is  to  be  measured  in  the  project,  (c) 
logistical  issues  associated  with  planned  data  collection  activities,  and  (d) 
strategies  for  ensuring  that  the  products  developed  are  used  for  the  maximum 
benefit  of  the  Army. 

The  issues  associated  with  the  planned  data  collection  were  the  focus  of 
the  meeting,  particularly  problems  associated  with  getting  sufficient  sample 
sizes  and  recommendations  for  overcoming  these  obstacles.'  Specifically,  we 
were  concerned  that  our  troop  support  requests  would  be  inaccurate  due  to 
difficulty  in  projecting  future  locations  of  target  soldiers;  that  we  would 
have  difficulty  getting  individual  soldiers  to  testing  because  soldiers  were 
shifting  locations  and  leaving  the  Army;  that  we  would  have  limited  access  to 
soldiers  on  training  or  alert  status,  and  no  control  over  soldiers  on  annual, 
sick,  or  emergency  leave;  and  that  there  were  special  difficulties  testing 
target  soldiers  in  USAREUR  because  soldiers  are  widely  dispersed,  few 


It  is  important  to  note  that  the  Action  Officers'  Working  Group  met  5 
months  before  the  events  of  Desert  Shield/Desert  Storm, 
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locations  can  support  the  equipment  requirements,  and  taskings  were  done  at 
the  Corps  level.  Additional  anticipated  problems  caused  by  projected  down¬ 
sizing  in  response  to  the  current  events  in  Eastern  Europe  included  the 
geographical  shifting  of  soldiers  out  of  Germany,  low  reenlistment  rates,  and 
budget  cuts. 

The  Group  concurred  with  our  plans  to  improve  the  sample  sizes, 
including  increasing  the  scope  of  testing  by  testing  soldiers  who  have 
predictor  data,  first-tour  data,  or  both;  testing  at  additional  sites  (e.q., 
Fort  Drum,  Military  District  of  Washington/Aberdeen  Proving  Ground,  Korea); 
increasing  the  minimum  sample  size  requirements  for  high  density  MOS  (e.g., 
118,  13B,  91A).  We  also  proposed  to  increase  tne  testing  efficiency  by 
improving  soldier  tracking  methods;  seeking  increased  cooperation  from 
military,  to  include  flexibility  on  troop  support  requests,  tasking  at  level 
of  individual  soldiers  rather  than  at  the  unit  level,  and  seeking  more 
favorable  resolutions  of  scheduling  conflicts;  and  eliminating  hands-on 
testing  for  some  MOS  where  the  sample  sizes  would  be  small  and  the  equipment 
, '  Tsts  both  high  and  burdensome. 


,  In-Progress  Reviews  (IPR) 

First  IPR.  Final  recommendations  for  the  experimental  battery  predictor 
composites  were  made;  the  development  of  a  set  of  alternative  composite  stores 
for  ABLE  was  summarized;  the  procedures  for  computing  predictor  composite 
scores,  including  the  imputation  process  that  was  used,  were  described;  the 
initial  results  for  prediction  of  EOT  criteria  with  Experimental  Battery 
predictor  composites,  including  the  alternative  ABLE  composites,  were 
reported;  the  development  of  the  second-tour  performance  model  was  described, 
along  with  the  initial  results  for  predicting  the  elements  of  the  model  with 
ASVAB  and  ABLE  scores;  the  initial  data  screening  process  was  described  for 
the  longitudinal  validity  criterion  data  file,  as  was  the  status  of  ratings, 
administrative  measures,  school  knowledge  tests,  job  knowledge  tests,  hands-on 
tests,  and  data  from  the  Multipurpose  Arcade  Combat  Simulator  (MACS).  A 
representative  from  the  Defense  Manpower  Data  Center  (DMDC),  Personnel  Testing 
Division  described  upcoming  activities  relevant  to  ASVAB  updates  and  other 
revisions  to  Department  of  Defense  testing/classification  activities.  A 
summary  of  the  meeting  follows. 

Use  of  prior  methods.  Several  discussions  focused  on  methods  of 
handling  or  analyzing  data  as  it  had  been  done  during  Project  A.  The  analysis 
teams  generally  followed  the  procedures  developed  inaarlicr  phases  of  the 
research  program.  The  group  agreed  that  this  was  the  best  way  to  proceed, 
making  comparisons  between  current  and  prior  analyses  as  meaningful  as 
possible.  It  was  agreed  that  where  new  methods  are  indicated  by  empirical 
findings  from  earlier  or  current  project  analyses  or  by  advances  in  technique, 
then  the  new  methods  would  be  used  and  the  reason  for  the  change  in  method 
documented. 

Differences  between  "screened  in"  and  "screened  out"  groups  for  the 
ABLE .  Concerns  were  expressed  that  the  screening  procedures  might  produce 
important  differences  between  groups  passing  the  screens  and  groups  failing 
the  screens.  The  groups  should  be  compared  on  variables  like  race,  gender, 
and  ASVAB  scores.  The  intent  is  not  to  change  the  screening  procedures  for 
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research  purposes,  but  to  be  prepared  to  answer  questions  in  the  future, 
especially  if  implementation  becomes  an  issue. 

Sets  of  ABLE  composite  scores.  End-of-training  and  second-tour 
performance  validation  analyses  revealed  very  little  difference  between  the 
three  sets  of  ABLE  composites.  The  view  was  expressed  that  achieving  equal 
validities  with  two-thirds  of  the  items  weighs  in  on  the  side  of  the  shorter 
version.  However,  the  most  telling  evidence  will  come  from  the  longitudinal 
design,  and  the  group  thought  it  best  to  await  that  evidence  before  making  any 
decisions  on  the  most  useful  set  of  composite  scores. 

Second-tour  performance  model.  The  recommended  second-tour  performance 
model  contained  six  factors  labeled  as  substantive  and  two  labeled  as  method. 

A  recommendation  was  made  that  we  attempt  to  confirm  a  "five  substantive  and 
three  method"  factor  model  with  the  Simulation  (role  play)  measures  treated  as 
a  method  factor. 

Additional  analyses  for  second-tour  validation.  Three  specific  types  of 
additional  analyses  were  suggested:  (a)  look  at  the  Situational  Judgment  Test 
separately,  (b)  use  ABLE  composites  separately,  and  (c)  use  a  priori  weights 
for  the  predictors  (from  the  Synthetic  Validity  project  (Wise  et  al.,  1991) 
and  Project  A  first-tour  analyses). 

Second  IPR.  The  following  topics  were  discussed:  end-of-training 
validation  results  for  LVI,  CVII  validation  results,  CVI  and  CVII  relation¬ 
ships,  LVI  model  confirmation  and  initial  validation  analyses,  and  an 
attrition  analyses  working  group.  A  summary  of  each  follows. 

End-of-training  validation  results  for  LVI.  The  end-of-training 
validation  results  for  the  LVI  sample  were  presented.  There  was  some 
discussion  about  the  meaning  and/or  explanation  of  the  significant  intercept 
differences  in  these  analyses:  Is  this  "good"  or  "bad"?  What  is  the  source 
of  the  difference(s)?  Regression  plots  and  criterion  and  predictor  means  and 
standard  deviations  could  be  examined  to  shed  more  light  on  the  "cause"  of  the 
finding,  but  the  implication  will  remain  the  same  in  any  case--that  separate 
intercepts  should  be  used  for  each  MOS.  This  seems  to  be  "neutral,"  in  that 
it  is  in  line  with  current  practice  and  does  not  appear  to  have  any  negative 
implications  for  the  ASVAB  or  other  predictors. 

CVII  validation  results.  A  notation  was  made  that  some  of  the  critical 
CVII  analyses  should  be  redone  with  MOS  IIB  (Infantryman)  removed.  This 
should  make  results  for  Core  Technical  Proficiency  and  General  Soldiering 
Proficiency  more  consistent.  Zero  order  correlations  should  be  included  with 
regression  weights  for  the  CVII  hierarchical  analyses,  and  better  summary 
tables  are  needed  for  those  analyses.  The  summary  of  the  validity 
generalization  analy'^is  should  show  what  happens  when  the  predictors  in  the 
equation  are  held  constant  and  only  the  weights  vary  (in  contrast  to  allowing 
both  the  predictors  and  weights  to  vary).  It  was  also  noted  that  showing 
statistical  significance  tests  for  the  incremental  validities  and  for 
differences  produced  by  using  non-optimal  weights  would  be  desirable. 

CVI  and  CVII  relationships.  Analyses  were  presented  showing  the 
relationship  between  CVI  and  CVII  criterion  scores  and  the  incremental 
validities  of  the  CVI  criteria  over  the  prediction  achieved  by  the  ASVAB  and 
Irial  Battery,  as  weighted  by  CVI  least-squares  weights.  There  was  some 
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discussion  of  the  nature  of  range  restriction  attributable  to  the  reenl istment 
process.  Many  soldiers  are  declared  ineligible  who  might,  in  fact,  be 
eligible  if  they  expressed  a  desire  to  reenlist.  Furthermore,  all  require¬ 
ments  for  eligibility  can  be  waived  at  the  local  level.  The  murkiness  of  the 
process  makes  it  difficult  to  decide  how  to  correct  range  restriction  for  the 
CVI/CVII  samples. 

LVI  model  confirmation  and  initial  validation  analyses.  The  issue  of 
using  zero  uniqueness  values  in  the  confirmatory  model  analyses  was  questioned 
and  discussed.  It  was  agreed  that  there  was  no  satisfactory  method  of  fixing 
or  estimating  the  uniqueness  estimates  when  they  became  negative  if  left  free. 
It  was  decided  that  the  model  confirmation  analyses  would  be  rerun  with  1 
minus  the  squared  multiple  correlation  as  the  uniqueness  estimate,  to  see 
whether  any  appreciable  differences  occurred  with  this  procedure,  as  compared 
to  the  usp  f  zeroes. 

There  was  considerable  discussion  of  the  different  results  obtained  for 
the  Effort  and  Leadership  criterion  in  the  LVI  sample  in  contrast  to  the  CVI 
sample.  Basically,  this  criterion  is  better  predicted  by  cognitive  measures 
and  less  well  predicted  by  ABLE  measures  in  the  LVI  sample  wiien  compared  to 
the  CVI  sample.  This  finding  is  complicated  by  some  changes  in  the  criterion 
construct  itself  and  some  changes  in  the  makeup  of  the  ABLE  composites. 

Attrition  analyses  working  group.  A  working  group  was  formed  to  work  on 
attrition  analyses.  The  group  will  examine  the  methods  used  by  others  to  form 
criterion  scores  and  conduct  the  analyses,  as  well  as  examining  the  use  of 
survival  analysis  techniques. 


SUMMARY  OF  SCIENTIFIC  WORK  FOR  YEAR  ONE 

As  described  in  the  first  annual  report  (Campbell  &  Zook,  1990),  the 
objectives  of  the  project's  first  year  were  focused  on  developing  a  full 
design  for  the  data  base  and  on  analyzing  basic  scores  for  (a)  the  final 
version  of  the  Experimental  Predictor  Battery  (EB),  (b)  the  End-of-Training 
(EOT)  performance  measures,  and  (c)  the  second-tour  criterion  measures  used  to 
assess  NCO  performance  in  the  second-tour  Concurrent  Validation  (CVI I)  sample. 
The  data  from  the  End-of-Training  (EOT)  and  second-tour  Concurrent  Validation 
(CVI  I)  performance  assessment  were  also  used  to  formulate  both  a  model  of 
training  performance  and  a  model  of  second-tour  (junior  NCO)  job  performance. 
That  is,  the  basic  scores  from  the  individual  performance  measures  were 
aggregated  into  factor  scores  that  represented,  as  well  as  possible,  the  major 
components-  or  latent  structure,  of  training  performance  and  second-tour  job 
performance. 

By  the  end  of  year  one,  the  data  collection  for  the  Longitudinal 
Validation  first- tour  performance  assessments  had  been  completed,  but  the  data 
cleaning  and  editing  were  still  in  progress  and  the  analysis  of  the  LVI 
performance  measures  had  not  yet  begun. 

Data  Base  Design 

As  described  in  the  first-year  annual  report,  the  Career  Force  data  base 
design  allows  access  at  any  level  of  score  aggregation.  The  report  describes 
each  variable  and  the  amount  of  information  that  is  available.  The  data  are 
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stored  at  the  NIH  computing  facility  and  are  accessed  via  a  secure  system  that 
requires  prior  approval  by  the  Army. 

The  data  base  also  includes  data  from  the  following  operational  files 
maintained  by  the  Army: 

-  Applicant/Accessions  Data 

-  Training  Data 

-  Skill  Qualification  Test  Data 

-  Enlisted  Master  Data  Center  Cohort  Data 

-  World-Wide  Locator  Data 

Continuous  file  updates  are  made  only  for  the  Enlisted  Master  File  (EMF), 
which  is  updated  on  a  quarterly  basis  for  each  individual  in  all  Project  A  and 
Career  Force  Project  cohorts. 

We  continue  to  believe  that  the  entire  data  base  will  be  an  extremely 
valuable  resource  for  Army  personnel  management  for  many  years  in  the  future. 


Basic  Scores  for  the  Experimental  Battery 

During  year  one,  much  effort  was  devoted  to  analyzing  the  data  that  had 
been  obtained  by  administering  the  Experimental  Predictor  Battery  to 
approximately  50,000  new  accessions  in  the  Longitudinal  Validation  sample.  A 
number  of  data  editing  procedures  were  compared  and  evaluated,  and  great  care 
was  taken  to  maximize  data  quality  for  the  information  that  was  entered  into 
the  final  data  file.  The  psychometric  properties  and  subgroup  differences  for 
each  measure  were  analyzed,  and  a  series  of  exploratory  and  confirmatory 
analyses  were  conducted  to  identify  the  basic  predictor  scores  within  each 
domain  that  would  be  used  in  the  validation  analyses. 

The  final  array  of  tests  in  the  Experimental  Battery  and  the  constructs 
they  are  intended  to  measure  are  shown  in  Figure  1.5.  The  31  basic  scores 
that  are  obtained  from  the  specific  test  indicators  are  shown  in  Figure  1.6 
(Campbell  &  Zook,  1990). 

There  was  a  very  high  degree  of  consistency  between  the  Concurrent 
Validation  and  the  Longitudinal  Validation  in  terms  of  the  factor  structures 
of  the  various  measures.  The  resulting  definitions  of  the  basic  predictor 
scores  to  be  used  in  the  validation  analyses  were  quite  similar. 

Basic  Scores  for  the  End-of-Training  Measures 

During  year  one,  the  data  from  the  School  Knowledge  Test  and  the  seven 
training  performance  rating  scales  were  analyzed  in  terms  of  their 
psychometric  properties  and  factor  structure.  Confirmatory  techniques  were 
used  to  ^■'entify  the  "model"  of  training  performance  that  best  represented  the 
covar'h IS  among  the  observed  measures.  That  is,  an  a  priori  set  of 
alterna.'  vj  models  was  proposed  and  evaluated  in  terms  of  the  degree  to  which 
they  fit  the  data.  In  the  end  six  basic  scores  were  proposed,  two  based  on 
the  knowledge  test  and  four  based  on  the  rating  scales.  A  brief 
characterization  of  the  six  scores  is  given  in  Figure  1-7. 
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Test/Maasure 


Construct 


Paper-- and- Pend  J  Spatial  Tests 

1 

Assembling  Objects 

Object  Rotation 

Maze 

Orientation 

Map 

Reasoning 

Spatial  Visualization-Rotation 

Spatial  Visualization-Rotation 

Spatial  Visualization-Scanning 

Spatial  Orientation 

Spatial  Orientation 

Induction 

Computer-Administered  Tests 

Simple  Reaction  Time 

Choice  Reaction  Time 

Short-Term  Memory 

Perceptual  Speed  and  Accuracy 
Target  Identification 

Target  Tracking  1 

Target  Shoot 

Target  Tracking  2 

Number  Memory 

Cannon  Shoot 

Reaction  Time  (Processing  Efficiency) 
Reaction  Time  (Processing  Efficiency) 
Short-Term  Memory 

Perceptual  Speed  and  Accuracy 

Perceptual  Speed  and  Accuracy 
Psychomotor  Precision 

Psychomotor  Precision 

Multilimb  Coordination 

Number  Operations 

Movement  Judgment 

Temperament,  Interest,  and  Job  Preference  Measures 

Assessment  of  Background 
and  Life  Experiences  (ABLE) 

Adjustment 

Oependabi 1 ity 

Achievement 

Physical  Condition 

Leadership  (Potency) 

Locus  of  Control 

Agreeableness/L  ikabi 1 ity 

Army  Vocational  Interest 

Career  Examination  (AVOICE) 

Realistic  Interest 

Conventional  Interest 

Social  Interest 

Investigative  Interest 

Enterprising  Interest 

Artistic  Interest 

Job  Orientation  Blank  (JOB) 

Job  Security 

Serving  Others 

Autonomy 

Routine  Work 

Ambit ion/Achievement 

Figure  1.5.  Experimental  predictor  battery  tests  and  relevant  constructs. 
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EOT  RATING  SCALE  BASED  SCORES 

1) 

EFFORT  AND  TECHNICAL  SKILL  (ETS) 

Technical  Knowledge/Skill: 

How  effective  is  each  soldier  in 
acquiring  job/soidiering  knowledge 
and  skill? 

Effort: 

How  effective  is  each  soldier  in 
displaying  extra  effort? 

2) 

Maintaining  Personal  Discipline  (HPD) 

Following  Regulations 
and  Orders: 

How  effective  is  each  soldier  in 
adhering  to  regulations,  orders,  and 

SOP  and  displaying  respect  for 
superiors? 

Self  Control : 

How  effective  is  each  soldier  in 
controlling  own  behavior  related  to 
aggressive  acts? 

3) 

Physical  Fitness  and  Military 
Bearing  (PF8) 

Military  Appearance: 

How  effective  is  each  soldier  in 
maintaining  proper  military 
appearance? 

Physical  Fitness: 

How  effective  is  each  soldier  ii; 
maintaining  military  standards  of 
physical  fitness? 

4) 

Leadership  Potential  (LEAD): 

Leadership  Potential: 

Evaluate  each  soldier  on  his  or  her 
ootential  effectiveness  as  a  leader. 

Oc  not  necessarily  rate  on  the  basis 

of  present  performance. 

EOT  KNOWLEDGE 

TEST  BASED  SCORES 

5) 

Basic  Knowledge  Score: 

Items  measuring  knowledge 
requirements  common  to  all  MOS. 

6) 

Technical  Knowledge  Score: 

Items  measuring  technical  knowledge 
requirements  specific  to  each  MOS. 

Figure  1.7.  Composite  scores  that  reflect  End-of-Training  performance  factors. 
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These  six  scores  will  serve  both  as  criterion  measures  (for  the 
Experimental  Battery)  and  as  predictors  (of  first-tour  and  second-tour  job 
performance)  in  later  validation  analyses. 

Development  of  Second-Tour  Performance  Scores  (CVII) 

The  performance  measures  used  in  the  CVII,  and  their  development,  have 
been  described  in  detail  in  previous  reports  (Campbell,  1991;  Campbell  &  Zook, 
1991).  First-tour  measures  were  revised  for  use  with  second-tour  personnel 
and  new  measures  reflecting  the  unique  components  of  second-tour  jobs  were 
added.  A  summary  description  of  the  specific  measures  is  given  below. 

Rating  Scales 

On  the  basis  of  second-tour  critical  incident  analyses,  the  Army-wide 
Behavioral ly  Anchored  Ratings  Scales  (BARS)  and  MOS-specific  BARS  wei e  revised 
and  scales  having  to  do  with  leadership  and  supervision  were  added.  Further, 
based  on  job  analysis  data,  seven  new  scales  pertaining  to  supervision  and 
leadership  responsibilities  were  also  added.  A  full  list  of  the  Army-wide 
rating  scales  is  shown  below.  Not  shown  are  the  MOS  BARS  for  each  MOS,  which 
were  revised  to  reflect  second-tour  performance  demands,  and  the  Combat 
Prediction  Scales,  which  were  retained  in  the  same  form  they  were  used  in  LVI, 
but  which  were  not  administered  to  female  NCOs. 

Army-Wide  Behavior  Scales  (CVII): 

1.  Demonstrating  Technical  Knowledge  and  Skill 

2.  Demonstrating  Effort 

3.  Supervising  Subordinates 

4.  Following  Regulations  and  Orders 

5.  Demonstrating  Integrity 

6.  Training  and  Development  of  Subordinates 

7.  Maintaining  Equipment 

8.  Physical  Fitness 

9.  Self-Development 

10.  Showing  Consideration  for  Subordinates 

11.  Demonstrating  Appropriate  Military  Bearing 

12.  Demonstrating  Appropriate  Self-Control 

Additional  Leadership  Scales  (CVII): 

13.  Serving  as  a  Role  Model 

14.  Communication  with  Subordinates 

15.  Personal  Counseling 

16.  Monitoring  Subordinate  Performance 

17.  Organizing  Missions/Operations 

18.  Personnel  Administration 

19.  Performance  Counseling 

General  Scales: 

20.  Overall  Effectiveness 

21.  Senior  NCO  Potential 


A  new  paper-and-penci 1  measure  of  supervisory  judgment  was  developed  by 
describing  prototypical  judgmenc  situations  and  asking  the  respondent  to 
select  the  most  appropriate  and  the  least  appropriate  course  of  action.  The 
situation  descriptions  and  the  scoring  keys  were  refined  through  extensive 
subject  matter  expert  (SME)  judgments. 

Supervisory  Simulation  Exercise 

These  measures  were  developed  to  assess  NCO  performance  in  job  areas 
that  were  judged  to  be  best  assessed  through  the  use  of  interactive  exercises. 
The  simulations  were  designed  to  evaluate  performance  in  counseling  and 
training  subordinates.  A  trained  evaluator  (role  player)  played  the  part  of  a 
subordinate  to  be  counseled  or  trained  and  the  examinee  assumed  the  role  of  a 
first-line  supervisor  who  was  to  conduct  the  counseling  or  training. 

Evaluators  also  scored  the  examinee's  performance,  using  a  standard  set  of 
rating  scales. 

Here  are  brief  descriptions  of  the  three  simulation  exercises: 

-  Personal  Counseling  Simulation:  m  PFC  is  exhibiting  declining  job 
performance  and  personal  appearance.  Recently,  the  PFC’s  wall 
locker  was  left  unsecured.  The  supervisor  has  decided  to  counsel 
the  PFC  about  these  matters. 

-  Disciplinary  Counseling  Simulation:  There  is  convincing  evidence 
that  the  PFC  lied  to  ret  out  of  coming  to  work  today.  The  PFC  has 
arrived  late  to  work  on  several  occasions  and  has  been  counseled 
for  lying  in  the  past.  The  PFC  has  teen  instructed  to  report  to 
the  supervisor's  office  immediately. 

-  Training  Simulation:  The  commander  will  be  observing  the  unit 
practice  formation  in  3C  minutes.  The  private,  although  highly 
motivated,  is  experiencing  problems  with  the  hanu  salute  and  about 
face. 

For  each  exercise,  examinee  performance  was  evaluated  on  3-point  rating 
scales  reflecting  specific  behaviors  tapped  by  the  exercises  and  a  5-point 
overall  effectiveness  rating  scale. 

Factor  analyses  of  the  ratings  data  suggested  that  each  simulation  could 
be  scored  ir,  terms  of  the  content  of  the  NCO's  behavior  (i.e.,  did  he  or  she 
do  or  say  the  right  things)  and  the  process .  or  style,  with  which  the 
counseling  steps  were  carried  out. 

Administrative  Measure 


The  self-report  Personnel  File  Form  (PFr)  used  in  LVI  was  modified 
slightly  for  use  with  second  tour  and  six  administrative  indices  were 
obtained. 
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Job  Knowledge  and  Hands-On  Measures 

The  content  of  each  of  these  measures  was  revised  on  the  basis  of  the 
second-tour  job  analyses  and  the  revised  instruments  were  subjected  to 
extensive  SME  review.  Comprehensive  analyses  of  alternative  aggregations  of 
item  and  scale  scores  from  both  of  these  measures  resulted  in  the  adoption  of 
a  basic  (Army-wide)  and  technical  (MOS-specif ic)  score  for  each  of  them. 

Final  Array  of  Second-Tour  Basic  Performance  Scores 

After  extensive  analyses  of  their  psychometric  properties  and  factor 
structures,  based  on  CVII  data,  the  final  array  of  basic  second-tour 
performance  scores  was  as  shown  in  Figure  1.8.  There  were  22  basic  scores. 

It  is  this  array  of  scores  that  became  the  basis  for  the  second- tour 
performance  modeling  analysis. 


Development  of  Second-Tour  Performance  Model 

The  basic  CVII  performance  scores  served  as  input  to  the  development  of 
a  latent  structure  model  for  second-tour  performance.  Based  on  a  consensus  of 
the  project  staff,  three  major  alternatives  could  be  used  to  explain  the 
observed  correlations.  Consequently,  the  competing  models  that  were  evaluated 
for  comparative  goodness  of  fit,  using  the  LISREL  VI  program  (Joreskog  & 
Sorbom,  1986),  were  the  following. 

(1)  First-Tour  Model:  Includes  five  substantive  and  two  methods 
factors,  with  the  SOT  and  Simulation  variables  all  loading  on  the 
Effort  and  Leadership  factor. 

(2)  Leadership  Factor  Model:  Includes  a  sixth  substantive  factor  with 
the  SJT,  Simulation,  and  Leadership  Rating  factor  variables  all 
loading  on  this  factor.  This  model  was  evaluated  with  and  without 
a  separate  Role-Play  "methods"  factor. 

\  _ 

(3)  Training  and  Counseling  Factor  Model;  Includes  a  sixth 
substantive  factor  with  just  the  Simulation  variables.  No 
separate  Role-Play  methods  factor  could  be  estimated  under  this 
model . 

Of  the  three  models,  the  Training  and  Counseling  Factor  Model  provided 
the  closest  fit  to  the  observed  data.  The  Simulation  exercise  scores  showed  a 
good  deal  of  internal  consistency,  but  had  very  low  correlations  with  any  of 
the  other  performance  measures.  Consequently,  any  model  that  included  a 
factor  with  loadings  for  both  Simulation  variables  and  other  performance 
variables  did  not  provide  a  reasonable  ^it  to  the  data  Us  either  the 
consistency  among  the  Simulation  exercises  was  underestimated  or  their 
correlations  with  other  measures  were  overestimated).  A  result  of  consider¬ 
able  interest  was  that  the  SJT  (a  paper-and-penci 1  measure)  fit  best  with  the 
Effort  and  Leadership  factor,  in  spite  of  the  method  variance  involved. 

The  basic  scores  that  have  been  used  to  represent  the  latent  variables 
are  as  shown  in  Figure  1.9.  For  validation  analysis  purposes,  the  six 
substantive  factor  scores  are  obtained  by  standardizing  and  summing  the  basic 
scores  within  each  factor. 
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Hands-On  Job  Sample  Test 

1.  MOS-specific  task  performance  score 

2.  Common  task  performance  score 

Job  Knowledge  Test 

3.  MOS-specific  task  knowledge  score 

4.  Common  task  knowledge  score 

Army-Wide  Rating  Scales 


5.  Leadership/supervision  composite 

6.  Technical  proficiency  and  effort  composite 

7.  Maintaining  personal  discipline  composite 

8.  Physical  fitness  and  military  bearing  composite 

MOS-Soecific  BARS  Scales 

9.  One  overall  MOS  BARS  composite  score 
Combat  Performance  Prediction  Scales 

10.  One  overall  combat  scale  composite 
Personnel  File  Form 


11.  Awards  and  Commendations 

12.  Articles  15/Flag  Actions 

13.  Physical  Qualification 

14.  M16  Qualification 

15.  Number  of  Military  Training  Courses 

16.  Promotion  Rate 

Situational  Judgment  Test 

17.  One  score  obtained  by  subtracting  the  total  "ineffectiveness 
score  from  the  total  "effectiveness"  score 

Supervisory  Simulation  Exercises 

18.  Personal  Performance  Counseling:  Interaction  Process 

19.  Personal  Performance  Counseling:  Interaction  Content 

20.  Discipline  Problem  Counseling:  Interaction  Process 

21.  Discipline  Problem  Course  1 ing:  Interaction  Content 

22.  One-on-One  Training:  Total  Composite  Score 


Figure  1.8.  Summary  list  of  second-tour  basic  criterion  scores. 


Latent  Variables  in  the  CVII  Performance  Model 

•  Core  Technical  Proficiency  (CTP) 

-  Job-Specific  Hands-On 

-  Job-Specific  Knowledge 

•  General  Soldiering  Proficiency  (GSP) 

-  Common  Hands-On 

-  Common  Job  Knowledge 

•  Effort  and  Leadership  (ELS) 

-  Awards  and  Certificates 

-  Training  Courses 

-  Grade  Deviation  Score 

-  Army-Wide  BARS  Leadership  Rating 

-  Army-Wide  BARS  Technical  Rating 

-  MOS  BARS  Average  Rating 

-  Situational  Judgment  Test 

•  Personal  Discipline  (MPO) 

-  Articles  15,  Flag  Actions  (reversed) 

-  Army-Wide  BARS  Discipline  Rating 

•  Physical  Fitness/Hi  1 itary  Bearing  (PFB) 

-  Physical  Readiness  Score 

-  Army-Wide  BARS  Fitness/Bearing  Rating 

•  Training  and  Counseling  Subordinates  (TCS) 

-  Simulation  Exercise  -  Personal  Counseling  Content 

-  Simulation  Exercise  -  Personal  Counseling  Process 

-  Simulation  Exercise  -  Disciplining  Content 

-  Simulation  Exercise  -  Disciplining  Process 

-  Simulation  Exercise  -  Training 

•  Written  Methods  (WM) 

-  Job-Specific  Knowledge 

-  Common  Soldiering  Knowledge 

-  Situational  Judgment  Test 

•  Ratings  Methods  (RM) 

-  Four  Army-Wide  BARS  Dimensions 

-  MOS  BARS  Average 


Figure  1.9.  Relationship  of  specific  variables  to  overall  factors  in  the  CVII 
performance  model. 


PROJECT  EFFORTS  FOR  YEAR  TWO 

The  remainder  of  this  report  describes  the  work  that  was  done  during  the 
second  year  of  the  Career  Force  Project.  Year  two  was  a  period  of  score 
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development,  model  building,  and  basic  validation  analyses  for  (a)  training 
performance  (EOT),  (b)  first-tour  performance  (LVI),  and  (c)  second-tour 
performance  (CVII). 

During  year  two  the  second-tour  longitudinal  data  collection  (LVI I) 
began  and  is  currently  ongoing.  The  nature  of  the  LVII  data  file  will  be 
described  in  the  annua'*  report  for  year  three. 


Objectives 

The  specific  objectives  for  the  second-year  annual  report  are  as 
fol lows. 

(1 )  Describe  the  basic  validation  analyses  for  the  prediction  of 
performance  in  training.  Last  year's  annual  report 
(Campbell  &  Zook,  1990)  described  the  development  of  the 
end-of-training  performance  scores.  These  six  scores  are 
the  criteria  against  which  the  Experimental  Predictor 
Battery  was  validated. 

( 2 )  Describe  the  development  of  basic  scores  for  the  longitudinal 
sample  first-tour  performance  measures.  The  nature  of  the  LVI 
sample  was  described  in  last  year's  report.  The  basis  for 
developing  the  LVI  basic  performance  score  was  the  parallel 
analysis  for  CVI,  which  was  described  in  the  Project  A  annual 
report  for  1987  (Campbell,  1989). 

(3)  Describe  the  replication/confirmation  of  the  first-tour 
performance  model  and  the  basic  Longitudinal  Validation  analyses 
for  the  Experimental  Predictor  Battery  against  first-tour 
performance.  These  analyses  use  the  LVI  sample,  which  was 
described  in  the  annual  report  for  year  one.  The  development  of 
the  final  basic  scores  for  the  Experimental  Predictor  Battery  was 
also  described  in  last  year's  report.  Included  in  this  year's 
work  was  a  special  set  of  analyses  that  focused  on  developing  a 
new  set  of  factor-analytic-based  scores  for  the  ABLE. 

( 4 )  Describe  the  basic  validation  analyses  for  the  prediction  of 
second-tour  performance,  using  the  CVII  sample.  The  parameters  of 
the  second-tour  Concurrent  Validation  sample  have  also  been 
described  previously.  For  most  of  the  sample,  the  available 
predictors  were  limited  to  the  ASVAB  scores  obtainable  from  the 
permanent  file  and  the  ABLE  which  was  given  concurrently  to 
approximately  one-half  of  the  total  sample.  Consequently,  the 
CVII  validation  analyses  must  be  regarded  as  a  prototype  for  the 
LVII  sample,  which  will  have  larger  Ns  and  a  more  complete 
predictor  set, 

( 5 )  Report  the  results  of  a  preliminary  analysis  of  the  prediction  of 
second-tour  performance  from  first-tour  predictors  and 
performance.  This  more  complete  data  set  was  available  only  for  a 
very  small  subsample  of  soldiers  from  the  CVI  sample  who  were  also 
assessed  on  the  CVII  second-tour  performance  measures. 
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Organization  of  This  Report 


Chapter  2  is  an  extension  of  the  analysis  of  the  LV  Experimental 
Predictor  Battery  scores.  It  deals  specifically  with  the  development  of  two 
alternative  sets  of  factor  scores  for  the  ABLE,  thus  producing  three 
alternative  scoring  procedures,  which  will  be  evaluated  in  subsequent 
chapters.  This  ABLE  composite  score  development  was  carried  out  by  Dr. 
Leonard  White  and  his  colleagues  at  the  Army  Research  Institute. 

Chapters  3  through  9  deal  with  various  aspects  of  the  basic  predictor 
validation  analyses  and  move  sequentially  from  the  prediction  of  training 
performance  to  first-tour  job  performance  to  second-tour  performance. 

Chapter  3  presents  the  basic  validation  analyses  for  the  prediction  of 
end-of-training  performance,  using  ASVAB  and  the  LV  Experimental  Predictor 
Battery. 

Chapter  4  describes  the  development  of  the  basic  scores  for  first-tour 
job  performance  based  on  the  data  from  the  performance  assessments  conducted 
in  the  LVI  sample.  Chapter  5  then  reports  the  data  editing  and  missing  data 
imputation  procedures  that  were  used  to  produce  a  final  data  file  for  the  LVI 
performance  measures.  Using  these  completed  files,  the  next  step  was  to 
conduct  confirmatory  analyses,  reported  in  Chapter  6,  to  assess  the  goodness 
of  fit  of  the  Concurrent  Validation  (CVI)  performance  model  to  the 
Longitudinal  Validation  data,  as  against  alternatives. 

Chapter  7  then  presents  the  results  of  the  validation  of  the  LV 
Experimental  Predictor  Battery  against  the  basic  LVI  performance  factors 
identified  in  Chapter  6.  Both  the  content  of  the  performance  factors  and  the 
predictive  validities  were  quite  consistent  from  CVI  to  LVI, 

Chapter  8  describes  the  validation  results  for  predicting  second-tour 
performance,  using  ASVAB  and  ABLE  as  predictors.  The  limitations  of  the 
sample  and  available  predictors  prevented  a  complete  analysis  of  the  validity 
of  the  Experimental  Battery  for  predicting  second-tour  performance,  but  the 
results  are  significant  and  very  encouraging. 

Finally,  Chapter  9  discusses  a  preliminary  analysis,  on  a  very  small 
sample,  of  the  validity  of  (a)  the  LV  Experimental  Battery,  (b)  end-of- 
training  performance,  and  (c)  first-tour  performance  for  predicting  second- 
tour  performance.  Even  though  the  number  of  people  who  had  complete  data  on 
all  predictors  was  very  small,  the  level  of  validity  and  the  degree  of 
convergent  and  discriminate  validity  were  substantial. 

In  sum,  the  Career  Force  Project  Second  Year  Annual  Report  describes  a 
sequence  of  analyses  based  on  four  different  samples:  (a)  the  Longitudinal 
Validation  predictor  sample  (LV),  (b)  the  Longitudinal  Validation  End-of- 
Training  Sample  (LVT),  (c)  the  Longitudinal  Validation  first-tour  performance 
measurement  sample  (LVI),  and  (d)  the  Concurrent  Validation  second-tour 
performance  measurement  sample  (CVII).  The  basic  parameters  of  these  samples 
are  as  described  above.  More  detailed  descriptions  of  specific  sample  sizes 
and  ineasures  are  given  in  the  appropriate  chapters.  Chapter  10  provides  a 
brief  summary  of  the  project's  second  year  and  describes  plans  for  future 
research  activities. 
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Chapter  2 

DEVELOPMENT  OF  COMPOSITE  SCORES  FOR  ASSESSMENT 
OF  THE  BACKGROUND  AND  LIFE  EXPERIENCES  (ABLE)  INSTRUMENT 

Leonard  A.  White 

This  chapter  describes  the  composite  scores  developed  for  the  ABLE 
administered  in  the  Longitudinal  Validation  (LV).  Combining  ABLE  content 
scales  into  a  smaller  number  of  interpretable  composites  has  practical 
advantages.  ABLE  composites  can  predict  job  performance  with  a  more 
manageable  number  of  test  scores.  Also,  such  composites  can  provide  more 
reliable  measures  of  underlying  constructs. 


ABLE  RATIONAL  SCALES  AND  COMPOSITES 

At  the  beginning  of  development  of  the  ABLE  instrument  in  1982, 
psychologists  regarded  temperament  measures  as  poor  predictors  of  job 
performance  (e.g.,  Guion  &  Cottier,  1965),  and  there  was  no  prevailing 
taxonomy  of  temperament  constructs.  However,  some  researchers  argued  that 
personality  was  described  by  five  dimensions  of  temperament,  known  as  the  Big 
Five  (Norman,  1963;  Goldberg,  1981).  The  Big  Five  are  Emotional  Stability, 
Agreeableness,  Cot;scientiousness,  Intel lectance,  and  Surgency.  Later,  Hogan 
(1982)  distinguished  a  Sociability  component  of  Surgency,  yielding  a  total  of 
six  temperament  dimensions. 

Using  Hogan's  six  dimensions  as  a  starting  point  for  a  taxonomic 
analysis.  Project  A  staff  reviewed  146  temperament  measures  and  sorted  them 
into  construct  categories.  Eighty  percent  of  the  scales  matched  one  of  the 
six  constructs  proposed  by  Hogan.  The  remaining  20  percent  of  the  scales  fit 
into  categories  of  Achievement,  Masculinity,  and  Locus  of  Control  (see  Kamp  & 
Hough,  1988  for  details  on  the  development  of  this  taxonomy).  To  determine 
relationships  between  these  temperament  constructs  and  job  performance, 

Project  A  researchers  reviewed  237  criterion-related  validity  studies 
conducted  between  1960  and  1984.  Results  showed  that  temperament  constructs 
predict  multiple  components  of  job  performance  and  training  success  (Hough, 
Eaton,  Dunnette,  Kamp,  &  McCloy,  1990). 

Based  on  the  results  of  this  literature  review,  10  temperament  scales 
were  developed  to  form  ABLE.  These  constructs  were  selected  as  the  most 
promising  for  predicting  performance  in  U.S.  Army  enlisted  occupational 
specialties.  In  addition,  four  validity  scales  were  added  to  detect 
inaccuracies  in  self-reports  of  temperament;  for  example,  the  Unlikely  Virtues 
scale  detects  social  desirability  response  bias  (i.e.,  "faking  good").  A 
self-report  measure  of  physical  condition  was  also  included  on  ABLE  (see 
Hough  et  al.,  1990,  for  more  information  on  the  development  of  ABLE). 

Project  A  staff  wrote  items  for  each  target  construct  that  sampled 
behaviors,  beliefs,  and  attitudes  thought  to  indicate  the  underlying  trait  in 
the  applicant  population.  They  also  assigned  items  to  specific  scales  prior 
to  data  analyses.  Subsequent  item  screening  involved  assessing  the 
gender/race  sensitivity  and  psychometric  characteristics  of  each  item.  Thus, 
ABLE  is  primarily  a  rationally  derived  measure,  but  has  been  modified  somewhat 
using  homogeneous  scaling  methods.  Table  2.1  presents  descriptive  statistics 
for  LV  cases  having  complete  ABLE  data  (screened  for  random  responding). 
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Table  2.1 


ABLE  Scale  Statistics  for 

Longitudinal 

Validation 

Sample 

Scale 

No.  of 
Items 

Mean 

SD 

Coefficient 

Alpha 

Content  Scales 

Self-Esteem 

12 

28.8 

4.00 

.79 

Work  Orientation 

19 

45.3 

6.25 

.87 

Energy  Level 

21 

50.6 

6.12 

.85 

Dominance 

12 

27.3 

4.69 

.84 

Traditional  Values 

11 

29.1 

2.92 

.64 

Conscientiousness 

15 

36.7 

4.15 

.73 

Nondelinquency 

20 

47.8 

5.59 

.79 

Emotional  Stability 

17 

40.2 

5.62 

.84 

Cooperativeness 

18 

44.5 

4.98 

.81 

Internal  Control 

16 

41.9 

4.42 

.77 

Physical  Condition 

6 

13.5 

3.04 

.82 

Validity  Scales 

Nonrandom  Response* 

8 

7.4 

1.23 

-- 

Uni ikely  Virtues 

11 

16.8 

3.38 

.63 

Poor  Impression 

23 

1.2 

1.66 

.64 

Self-Knowledge 

11 

26.3 

3.18 

.61 

Note.  N  =  39,904  unless  noted  otherwise. 

^Data  (N  =  48,378)  were  not  screened  for  randoir  responding. 


To  develop  ABLE  rational  composites,  Peterson  et  al.  (1992)  performed 
exploratory  and  confirmatory  (LISREL)  factor  analyses  on  the  correlations 
among  the  content  scale  scores.  The  best  fitting  confirmatory  model  contained 
six  factors.  In  this  model,  Dominance  clustered  with  Self-Esteem,  Energy 
Level,  and  Work  Orientation,  but  was  later  retained  as  a  separate  construct 
because  of  its  unique  potential  for  predicting  leadership. 

The  resulting  seven  temperament  constructs  and  associated  ABLE  scales 
are  shown  in  Table  2.2.  The  constructs  of  Dependability,  Dominance 
(Surgency),  Adjustment,  and  Cooperativeness  have  counterparts  in  the  Big  Five. 
Conversely,  Achievement  and  Internal  Control  are  not  in  the  Big  Five  taxonomy, 
but  were  among  the  strongest  predictors  of  job  performance  in  the  Project  A 
review  of  the  temperament  domain  (see  Hough,  1992  for  more  details  on  the 
relationship  of  ABLE  constructs  to  the  big  Five). 
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Table  2.2 


ABLE  Rational  Composites  and  Corresponding  Content  Scales 


_ Composite _ 

Achievement  Orientation 

Leadership  Potential 
Dependability 

Adjustment 
Cooperativeness 
Internal  Control 
Physical  Condition 


_ Scale _ 

Self-Esteem 
Work  Orientation 
Energy  Level 

Dominance 

Traditional  Values 

Conscientiousness 

Nondelinquency 

Emotional  Stability 

Cooperativeness 

Internal  Control 

Physical  Condition 


DEVELOPMENT  OF  ABLE  FACTOR  COMPOSITES 
Overview  and  Hypothesis 

There  are  several  methods  of  scale  construction  that  can  be  used  in 
developing  omnibus  personality  inventories  (Hase  &  Goldberg,  1967).  As  noted 
above,  a  rational/theoretical  approach  was  the  primary  method  used  in 
developing  ABLE,  An  alternative  procedure  is  an  empirical  method  of  scale 
construction  that  emphasizes  the  internal  variance  structir.e  of  a  set  of 
items.  Proponents  of  this  method  stress  the  importance  o^  homogeneous  scales. 
The  scale  development  approach  yielding  the  highest  criterion-related  validity 
would  probably  be  preferred.  However,  little  research  is  available  comparing 
the  validity  of  scales  developed  using  these  different  methods. 

Peterson  et  al.  (1992)  had  formed  the  rational  composites  by  combining 
ABLE  content  scales  into  meaningful,  but  net  necessarily  homogenous,  clusters; 
all  items  within  each  content  scale  were  assigned  to  the  same  construct.  For 
the  alternative  procedure,  it  was  hypothesized  that  the  homogeneity  of  ABLE 
composites  would  increase  if  we  allowed  items  from  any  given  content  scale  to 
be  keyed  for  different  constructs.  For  example,  we  expected  that  Energy  Level 
items  pertaining  to  chronic  fatigue  would  correlate  highest  with  Emotional 
Stability,  while  those  relating  to  energy  in  the  workplace  would  cluster  with 
Work  Orientation  items  (Taylor,  1953;  White,  Nord,  Mael,  &  Young,  in  press). 

In  addition,  Conscientiousness  items  were  expected  to  cluster  with  the 
Achievement  and  Dependability  constructs. 
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ABLE  Factor  Composites 


Internal  scale  construction  methods  were  used  to  increase,  through 
homogeneous  keying,  the  internal  consistency  of  ABLE  composites  and  to 
decrease  their  intercorrelations.  First,  we  performed  a  principal  factor 
analysis  with  varimax  rotation  on  the  199  ABLE  items  in  a  sample  of  39,904 
recruits  from  the  LV.  Eleven  factors  were  extracted  using  an  eigenvalue- 
greater-than-one  rule.  (Several  solutions  with  more  than  11  factors  were  run 
but  did  not  yield  additional  meaningful  factors.)  Our  examination  of  the  11- 
factor  solution  revealed  nine  interpretable  factors  that  accounted  for  65 
percent  of  the  variance. 

The  first  seven  rotated  factors  from  the  item-level  analyses  roughly 
corresponded  to  the  constructs  in  Table  2.2.  Leadership  Potential  and 
Achievement  emerged  as  separate  factors  in  the  analyses  of  ABLE  item  data.  As 
noted  earlier,  in  the  factor  analyses  of  ABLE  scales,  the  Dominance  and 
Achievement  scales  (e.g..  Work  Orientation)  loaded  on  the  same  factor,  but 
v/ere  placed  on  separate  constructs. 

Unlikely  Virtues  and  Self-Knowledge  comprised  the  remaining  two 
interpretable  factors.  We  had  included  items  from  the  validity  scales  to  test 
the  hypothesis  that  the  validity  and  content  scale  items  would  cluster  on 
distinct  factors.  Note  that  it  was  beyond  the  c  ,pe  of  these  analyses  to 
modify  the  validity  scales. 

Results  from  the  item  factor  analyses  were  used  to  form  seven 
preliminary  composites.  These  composites  contained  97  items  from  ABLE  content 
scakc  and  the  two  unique  items  from  Porr  Impression  that  clustered  with 
Adjustment.  Each  of  the  99  items  had  iiigh,  distinctive  loadings  (.34  or 
above)  on  one  factor  and  was  assigned  to  one  of  the  seven  temperament 
ct'.ipcsites.  Next,  correlations  between  the  remaining  70  items  and  the 
preliminary  factor  composites  were  examined.  We  then  assigned  each  of  these 
items  to  the  composite  with  which  it  had  the  highest  correlation.  One 
Traditional  Values  item  was  dropped  because  of  its  low  correlation  with  all 
the  factor  composites.  When  the  highest  item-scale  correlations  were  nearly 
equal,  the  item  content  was  examined  to  determine  the  scoring.  The  seven 
factor  composites  resulting  from  this  procedure  used  168  items  and  will  be 
called  the  ABLE-’68  composites.  In  all,  125  items  (74^)  were  assigned  in  the 
same  way  on  the  ABLE-168  and  rational  composites. 

The  item-scale  correlations  and  factor  loadings  for  items  on  ABLE-168 
were  varied,  and  some  items  did  not  improve  the  internal  consistency  of  the 
composite  for  which  they  were  keyed.  Therefore,  we  developed  a  second  set  of 
composites  by  applying  more  stringent  criteria  for  homogeneous  scoring  than 
was  used  for  the  ABLE-168. 

For  this  second  set  of  factor  composites,  we  retained  an  item  only  if  it 
correlated  at  least  .33  with  the  scale  for  which  it  was  assigned  and  had  a 
higher  correlation  with  its  own  composite  (by  .03)  than  any  other.  In 
addition,  several  items  that  added  only  minimally  to  internal  consistency  were 
dropped.  Using  these  decision  rules,  54  items  (32%  of  the  168)  v/ere  omitted 
in  an  iterative  fashion  based  on  discriminant  validity  and  their  contribution 
to  internal  consistency.  The  resulting  set  of  composites  had  a  total  of  114 
items  and  will  be  called  ABLE-114  composites.  Eighty-nine  of  these  items  were 
assigned  in  the  same  way  on  ABLE-114  and  the  rational  composites. 
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Results  and  Discussion 


The  three  scoring  methods  converged  to  yield  seven  similar  temperament 
constructs.  The  composites  measuring  the  same  constructs  were  very  highly 
correlated,  with  r  =  .88  to  1.0.  For  each  of  the  scoring  methods,  Table  2.3 
shows  the  correlations  among  the  composites  and  Unlikely  Virtues.  Coefficient 
alpha  reliabilities  appear  in  the  main  diagonal.  These  reliability  estimates 
were  nearly  identical  for  the  three  sets  of  composites,  ranging  from  a  median 
r  =  .83  to  .85. 

Table  2.3 

Correlations  Among  Alternative  ABLE  Composites  and  Unlikely  Virtues _ 

ABLE  Rational  Composites 


No.  of 


Composite 

Items 

1 

2 

3 

4 

5 

6 

7 

1.  Achievement 

52 

(93) 

2.  Leadership  Potential 

12 

.68 

(■84) 

3.  Dependability 

46 

38 

(88) 

4.  Adjustment 

17 

.71 

54 

.45 

(84) 

5.  Cooperatrveness 

18 

61 

.41 

.63 

55 

(.81) 

6.  Internal  Control 

16 

58 

.37 

56 

.48 

.49 

(77) 

7.  Physical  Condition 

6 

50 

.37 

.24 

J6 

.24 

.21 

(■82) 

8.  L'nlikely  Vinues 

11 

.39 

.22 

.44 

.28 

.37 

.19 

.16 

ABLE.168  facior.Basei)  Composites 


No.  of 


Composite 

Items 

1 

2 

3 

4 

5 

6 

7 

1.  Achievement 

42 

(.92) 

2.  Leadership  Potential 

23 

70 

(89) 

3  Dependabilin- 

33 

58 

.34 

(85) 

4.  Adjustment 

29 

.68 

67 

.44 

(.89) 

5.  Cooperaiiveness 

16 

.61 

.49 

56 

58 

(80) 

6.  Iniemal  Control 

17 

53 

35 

.48 

44 

.44 

(.78) 

7.  Ph;/sical  Condition 

8 

.47 

50 

.20 

46 

28 

19 

(83) 

8.  Unlikely  Vinues 

11 

44 

.25 

44 

.30 

.36 

21 

.17 

ABLE-114  Factor-Based  Composites 

Composite 

No.  of 

Items 

1 

2 

3 

4 

5 

6 

7 

1  Achievement 

28 

(.91) 

2.  Leadership  Potential 

19 

56 

(88) 

3.  Dependability 

21 

.45 

.24 

(82) 

4  Adjustment 

15 

53 

52 

36 

(84) 

5  Coopemtiveness 

10 

.46 

.35 

.48 

41 

(.75) 

6.  Internal  Control 

13 

52 

.33 

45 

.39 

.37 

(78) 

7  Physical  Condiiion 

8 

.47 

48 

15 

22 

.20 

(.83) 

8.  Unlikely  Vinues 

11 

42 

.23 

39 

.31 

.21 

.17 

N  =  3'J.904.  £<  OOl  for  all  correlaiions.  Coefficient  alpha  reliabilities  are  shown  in  paninthtses. 
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Thus,  the  internal  consistency  of  the  shorter  ABLE-114  composites  was 
apparently  maintained  by  selecting  the  most  homogeneous  items.  As  shown  in 
Table  2.3,  ABL£-114  composites  had  greater  discriminant  validity  than  both  the 
ABLE-168  factor  composites  and  the  rational  composites.  The  average 
correlation  among  the  composites  (off-diagonal  elements)  was  r  =.40  for  ABLE- 
114,  and  r  =  .47  for  the  rational  composites  and  ABLE-168.  Also,  the  three 
sets  of  ABLE  composites  had  similar  levels  of  correlation  with  the  Unlikely 
Virtues  scale,  with  mean  r  =  .28  to  .31. 

Table  2.4  shows  the  distribution  of  items  on  ABLE-168  and  ABLE-ii4  for 
each  of  the  ABLE  content  scales.  The  shaded  areas  in  the  table  mark  the 
scales  used  to  measure  the  rational  composites.  Those  items  appearing  in  the 
shaded  areas  are  common  to  both  the  rational  and  the  factor  composites,  with 
ABLE-114  items  shown  in  parentheses.  Items  outside  the  shaded  areas  were 
assigned  differently  on  the  rational  and  factor  composites.  For  example,  the 
21  Energy  Level  items  were  assigned  to  Achievement  on  the  rational  composites, 
but  only  13  of  these  items  were  assigned  to  Achievement  on  ABLE-168,  and  six 
were  scored  for  Achievement  on  ABLE-114.  On  the  factor  composites,  the 
remaining  Energy  Level  items  were  used  to  measure  Adjustment  and  Physical 
Condition. 

As  shown  in  Table  2.4,  there  is  much  overlap  between  the  rational  and 
factor  composites.  However,  approximately  25  percent  of  item  assignments  for 
the  factor  composites  were  different  from  those  used  for  the  rational 
composites.  Most  of  these  are  consistent  with  results  from  previous  research 
and/or  can  be  understood  on  the  basis  of  item  content. 

For  example,  the  Energy  Level  scale  measures  the  degree  of  energy  and 
enthusiasm  a  person  has,  as  characterized  by  not  tiring  easily,  taking  few 
breaks,  and  being  eager,  optimistic,  and  excited  (Hough  et  al.,  1990).  Items 
pertaining  to  energy  and  vigor  in  the  workplace  loaded  with  Work  Orientation 
and  were  used  to  measure  Achievement.  Those  indicating  general  lethargy, 
without  specific  reference  to  work,  aligned  with  Adjustment. 

Similarly,  Conscientiousness  items  relating  to  meeting  deadlines  and 
showing  responsibility  at  work  loaded  on  the  Achievement  composite. 
Conscientiousness  items  tapping  general  responsibility  and  organization,  but 
not  specific  to  the  workplace,  were  more  likely  to  cluster  on  Dependability. 
The  item  analyses  also  showed  a  close  correspondence  between  self-confidence 
in  one's  ability  (i.e.,  Self-Esteem)  and  a  reported  willingness  to  lead  (i.e., 
Dominance).  This  finding  is  consistent  with  previous  work  on  the 
relationships  between  Dominance,  Self-Esteem,  and  other  measures  (Hamilton, 
1971). 

In  sum,  we  developed  alternative  ABLE  composites  measuring  seven 
temperament  constructs.  The  114- item  form  is  shorter  and  has  higher 
discriminant  validity  than  the  other  two  sets  of  composites,  with  little 
apparent  loss  of  reliability.  Future  research  in  the  Career  Force  Project 
will  examine  the  criterion-related  validities  of  these  alternative  sets  of 
compos i tes. 


Distribution  of  ABLE  Scale  Items  on  ABLE-168  and  ABLE-114  Factor  Composites 
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Ntilc.  ABl,E-114  items  arc  shown  in  parentheses.  Shaded  areas  indicate  convergence  between  the  rational  and  factor  composites. 


Chapter  3 

PREDICTION  OF  PERFORHANCE  IN  TRAINING 
Scott  H.  Oppler,  Rodney  A.  McCloy,  and  Norman  G.  Peterson 


This  chapter  summarizes  the  results  of  tests  of  the  valiaity  of  the 
ASVAB  and  the  Project  A  Longitudinal  Validation  (LV)  Experimental  Battery  for 
predicting  training  performance  in  the  Army.  The  objectives  of  the  analyses 
described  here  were  as  follows: 


(1)  Compute  the  validities  for  ASVAB  and  Experimental  Battery  predictors 
against  peer  and  supervisor  ratings  and  against  written  measures  of 
training  performance. 

(2)  Compare  the  validities  of  four  alternative  sets  of  ASVAB  scores 

(9  ASVAB  subtests  vs.  4  ASVAB  factor  composites  vs.  AFQT  vs.  MOS- 
appropriate  Aptitude  Area  composites). 

(3)  Compare  the  validities  of  three  alternative  sets  of  ABLE  scores. 

(4)  Assess  the  incremental  validities  for  the  Experimental  Battery 
predictors  over  ASVA6. 

(5)  Compare  the  validation  results  for  the  prediction  of  peer  and 
supervisor  ratings  (controlling  for  number  of  raters  per  ratee). 


HETHOD 

Sample 

Of  the  50,237  recruits  administered  the  LV  Experimental  Battery  (paper- 
and-pencil  tests,  computer  tests,  or  both),  end-of-training  performance  data 
were  collected  from  27,378.  Following  final  editing  of  the  data,  there  were  a 
total  of  17,503  trainees  who  had  complete  predictor  (ASVAB  and  Experimental 
Battery)  and  criterion  (Job  Knowledge  Test  results  and  ratings  by  at  least  one 
peer  and  at  least  one  supervisor)  data.  A  breakdown  of  these  trainees  by  MOS 
is  shown  in  Table  3.i.  Asterisks  indicate  those  MOS  in  Batch  A,  who  were 
given  MOS-specific  testing  as  well  as  the  mere  general  measures. 

Note  that  the  trainees  in  MOS  i3G  (Cannon  Crewman)  have  been  separated 
into  two  groups,  13B-S  and  13B-T.  These  two  groups  correspond  to  two 
different  weapon  types  ("self-propelled"  and  "tc-wed")  for  which  the  trainees 
in  13B  could  have  been  trained.  Because  the  School  Knowledge  Tests  (described 
below)  administered  to  the  trainees  in  15B  differed  according  to  the  weapon 
type  used  in  training,  the  two  groups  of  trainees  were  treated  as  separate  MOS 
for  analyses  incorporating  scores  from  these  tests. 


Table  3.1 


Trainees  With  Complete  Predictor  Data  and  Complete  End-of -Training  Criterion 
Data,  by  NOS _ __________ 


- - -  _ 

MOS 

N 

IIB 

Infantryman 

1,798 

12B 

Coriidat  Engineer 

1,129 

13B 

Cannon  Crewman^ 

13B-S 

★ 

2,741 

13B-T 

★ 

605 

16S 

MANPADS  Crewman 

394 

19E  * 

M60  Armor  Crewman 

348 

19K  * 

Ml  Armor  Crewman 

1,251 

27E 

Tow/Oragon  Repairer 

33 

29E 

Comm. -Electronics  Radio  Repairer 

82 

31C  * 

Single  Channel  Radio  Operator 

423 

51B 

Carpentry/Hasonry  Specialist 

92 

54B 

NBC  Specialist 

418 

55B 

Ammunition  Specialist 

192 

63B  * 

Light-Wheel  Vehicle  Mechanic 

340 

67N 

Utility  Helicopter  Repairer 

190 

71l  - 

Administrative  Specialist 

913 

76Y 

Unit  Supply  Specialist 

563 

88M  * 

Motor  Transport  Operator 

403 

91A  •* 

Medical  Specialist 

2,447 

94B 

Food  Service  Specialist 

431 

95B  * 

Mi  1  itary  Police 

2,598 

96B 

Intelligence  Analyst 

112 

Batch  A 

* 

13,867 

Total 

17,503 

Mote.  "Conplete  predktor  data’  was  defined  as  A3VAB  scores  and  LV  Enperlmental  Ba:tery  results.  ’CorkiltLe 
EOT  criterior  data'  was  defined  as  scores  from  School  Knowledge  Tests  and  ratings  by  at  least  one 
peer  and  at  least  one  supervisor. 

*  indicates  Batch  A  MOS. 

•  Trainees  in  I3p  were  trained  (and  tested)  on  either  of  two  different  weapons  ("Self-propelled’  [S]  or 
’Towed’  [T]).  For  analyses  incorporating  School  Knowledge  test  data,  these  two  sc.i  of  trainees  were 
treated  as  belonging  to  separate  MOS. 


Measures 


Predictors 

The  predictor  scores  used  in  these  analyses  were  derived  from  the  ASVAB 
operaticnal ly  administered  when  these  soldiers  were  inducted,  and  from  the 
paper-and-pent i  1  and  computerized  tests  administered  in  the  Project  A  LV 
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r  oen'mental  Battery.  For  the  ASVAB,  four  types  of  scores  were  examined. 

I  »se  scores,  listed  in  Table  3.2,  include  the  nine  ASVAB  subtests,  the  four 
•'  VAB  factor  scores,  the  AFQT  (Armed  Forces  Qualification  Test),  and  the  MOS- 
.V  propriate  Aptitude  Area  composite  scores. 

T  bli  3.2 


;$  of  ASVAB  Scores  Used  in  End-of-Traininq  Validity  Analyses 


:ubtests 

General  Science 
Arithmetic  Reasoning 

Veroal  (Paragraph  Comprehension  +  Word  Knowledge) 

Numerical  Operations 

Coding  Speed 

Auto/Shop  Information 

Mathematical  Knowledge 

Mechanical  Comprehension 

Electronic  Information 


A  MB  Factor  Composites^ 

Technical  (Auto/Shop  Information,  Mechanical  Comprehension,  Electronics 
Information  subtests) 

Quantitative  (Mathematical  Knowledge,  Arithmetic  Reasoning  subtests) 

Verbal  (Word  Knowledge,  Paragraph  Comprehension,  General  Science  subtests) 
Speed  (Coding  Speed,  Number  Operations  subtests) 

•  led  Forces  Qualification  Test  (AFQT)^’ 

A;  itude  Area  Composites  (1  per  HOS) 

IIB:  CO  (Combat) 

12B:  CO  (Combat) 

13B:  FA  (Field  Artillery) 

16S;  OF  (Operators/Foods) 

19E:  CO  (Combat) 
i9K:  CO  (Combat) 

27E :  EL  (Electronics) 

29E:  £L  (Electronics) 

31C:  SC  ( Survei 1 lance/Communications) 

51B:  GM  (General  Maintenance) 

54B:  ST  (Skilled  Technical) 

55B:  GM  (General  Maintenance) 

63B:  MM  (Mechanical  Maintenance) 

67N:  MM  (Mechanical  Maintenance) 

71L:  CL  (Clerical) 

76Y:  Cl.  (rierical) 
fe8M:  Of  (Operators/Foods) 

91A;  S'l  (Skilled  Technical) 

94B:  or  (Operators/FooJs ) 

95B:  ST  (Skilled  Technical) 

96D:  ST  (Skilled  Technical) 


■  infynrvif ion  on  the  ASVA0  factor  analytes,  See,  for  c*anplc,  Anoberg,  StHUell,  Prestwoort,  &  Welsh, 

V,:  Mis,  MHcliell,  (, ration,  f,  Uitkj,  1982. 

omi'islte  of  fouf  AjVAB  subicsis  with  rruth  and  verbal  content. 


The  scores  derived  from  the  LV  Experimental  Batter^  are  listed  in  Table 
3.3.  With  one  exception,  these  scores  are  described  in  the  Career  Force  first 
annual  report  (Campbell  &  Zook,  1990).  The  exception  concerns  the  scores 
derived  for  the  ABLE.  Note  that  three  different  sets  of  ABLE  scores  are 
listed  in  Table  3.3.  The  first  set,  labeled  the  ABLE  Rational  Composites,  was 
derived  along  with  the  other  LV  predictor  co.mposites.  The  other  two  sets, 
labeled  ABLE-168  Factor  Composites  and  ABLE-114  Factor  Composites,  were  based 
on  results  of  factor  analyses  of  the  ABLE  items,  using  168  items  and  114  items 
respectively.  See  Chapter  2  for  a  description  of  the  development  of  the  ABLE 
factor  scores. 


Table  3.3 

Sets  of  LV  Experimental  Predictor  Battery  Scores  Used  in  End-of -Training 
Validity  Analyses _ 


Spatial  Composite 
Spatial 

Computer  Composites 
Psychomotor 
Perceptual  Speed 
Perceptual  Accuracy 
Number  Speed  and  Accuracy 
Basic  Speed 
Basic  Accuracy 
Short-Term  Memory 
Movement  Time 

JOB  Composites 

Autonomy 

High  Expectations 
Routine 

AVOICE  Composites 

Admini strative 
Audiovisual  Arts 
Food  Service 
Structure  1 /Machi nes 
Protective  Services 
Rugged/Outdoors 
Social 

Skil’ed  Technical 


ABLE  Rational  Composites 

Achievement  Orientation 

Adjustment 

Physical  Condition 

Internal  Control 

Co'^perativeness 

Dependabi 1 ity 

Leadership 

ABLE-168  Factor  Composites 

Locus  of  Control 
Cooperativeness 
Dominance 
Dependabi 1 ity 
Physical  Condition 
Stress  Tolerance 
Work  Orientation 

ABLE-114  Factor  Composites 

Locus  of  Control 
Cooperativeness 
Dominance 
Dependabi 1 ity 
Physical  Condition 
Stress  Tolerance 
Work  Orientation 


Criteria 


The  end-of-training  criterion  measures  consisted  of  written  tests  of 
school  knowledge  as  well  as  ratings  obtained  from  peers  and  supervisors. 

These  measures  were  described  in  detail  in  Campbell  and  Zook  (1990).  Briefly, 
a  written  test  of  school  knowledge  was  ueveloped  separately  for  each  MOS. 
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Based  on  results  of  confirmatory  factor  analyses,  two  scores  (referred  to  as 
Technical  and  Basic)  were  derived  from  these  tests  for  all  but  two  MOS,  IIB 
and  88M,  for  which  only  the  Technical  score  was  created.  Additionally,  a 
Total  score  was  created  for  each  MOS;  for  trainees  in  IIB  and  88M,  the 
Technical  score  served  as  the  Total  score. 

The  end-of-training  ratings  consisted  of  seven  scales  revised  from  the 
10  Behaviorally  Anchored  Rating  Scales  (BARS)  developed  to  assess  first-tour 
Army-wide  performance  (Pulakos  &  Borman,  1986).  Based  on  the  results  of 
exploratory  and  confirmatory  factor  analyses  (also  reported  in  Campbell  & 

Zook,  1990),  the  seven  scales  were  reduced  to  four  unit-weighted  composites: 
Effort  and  Technical  Skill  (ETS),  Maintaining  Personal  Discipline  (MPD), 
Physical  Fitness  and  Military  Bearing  (PFB),  and  Leadership  Potential  (LEAD). 
These  composites  were  computed  separately  for  peer  and  supervisor  ratings. 

For  most  analyses,  the  intention  was  to  obtain  the  most  reliable  ratings 
of  training  performance  possible.  In  those  instances,  ratings  were  averaged 
across  multiple  raters  (again,  separately  for  peer  and  supervisor  ratings), 
regardless  of  the  number  of  ratings  available  for  a  particular  ratee.  As 
indicated  before,  the  number  of  trainees  with  complete  predictor  data  and 
complete  criterion  data  (i.e..  School  Knowledge  Test  data  and  ratings  from  at 
least  one  peer  and  one  supervisor  rater)  was  reported  in  Table  3.1.  These 
trainees  were  used  for  the  analyses  described  in  Steps  A  and  B  oelow. 

For  some  analyses,  the  purpose  was  to  compare  validation  results  across 
rater  types  (i.e.,  peers  vs.  supervisors)  or  MOS.  Since  the  average  number  of 
ratings  per  ratee  varied  across  both  of  these,  this  element  was  controlled  by- 
constraining  the  number  of  peer  and  supervisor  ratings  for  each  trainee  in  trie 
sample  to  be  the  same.  Specifically,  as  described  in  Campbell  and  Zook 
(1990),  two  sets  of  peer  ratings  (one  labeled  "Peer  1,"  the  other  "Peer  2") 
and  one  set  of  supervisor  ratings  were  selected  randomly  for  each  trainee  in 
the  Table  3.1  sample  who  had  been  rated  by  at  least  that  many  raters.  (Note: 
Only  246  trainees  had  been  rated  by  more  than  one  supervisor.)  These  trainees 
(with  at  least  two  sets  of  peer  and  one  set  of  supervisor  ratings,  as  well  as 
complete  predictor  data)  were  used  in  analyses  comparing  the  validity  results 
associated  with  peer  and  supervisor  ratings  (see  step  C  below).  A  breakdown 
of  these  trainees  by  MOS  is  reported  in  Table  3.4. 


Procedure 

The  end-of-training  validation  analysis  procedure  consisted  of  the 
following  steps: 

A)  Using  the  sample  shown  in  Table  3.1,  multiple  correlations  between 
each  set  of  predictor  scores  and  each  set  of  criterion  scores  (the 
three  written  School  Knowledge  Test  scores,  the  four  "average"  peer 
rating  scores,  and  the  four  "average"  supervisor  rating  scores)  were 
computed  separately  by  MOS.  They  were  then  averaged  across  the 
Batch  A  MOS  and  across  all  MOS. 
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Table  3.4 


Trainees  With  Complete  Predictor 
Training  Ratings  From  Two  Peers 

Data,  School  Knowledge  Test  Data,  and  End-of- 
and  One  Supervisor,  by  HOS 

MOS 

N 

IIB  * 

1,759 

12B 

1,041 

13B  * 

3,305 

16S 

329 

19E  * 

341 

19K  * 

1,164 

27E 

33 

29E 

84 

31C  * 

377 

51B 

91 

54B 

370 

55B 

189 

63B  * 

288 

67N 

191 

71L  * 

843 

76Y 

524 

88M  * 

352 

91A  * 

2,373 

94B 

385 

95B  * 

2,519 

96B 

114 

Batch  A  * 

13,321 

Total 

16,672 

Note:  for  analyses  using  ratings  data  only,  the  distinction  between  trainees  in  13B-S  and  13B-T  was  not 

made. 

*  indicates  Batch  A  HOS. 
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1)  As  indicated  above,  the  ASVAB  predictor  set  was  represented  by 

a)  The  nine  ASVAB  subtest  scores; 

b)  The  four  ASVAB  factor  scores; 

c)  The  AFQT;  and 

d)  The  MOS-appropriate  Aptitude  Area  composite  score. 

2)  The  ABLE  predictor  set  was  represented  by  three  sets  of  scores: 

a)  The  seven  rational  scales  developed  for  the  Experimental 
Battery; 

b)  Seven  empirical  scales  developed  on  the  basis  of  a  factor 
analysis  that  retained  all  of  the  items  (and  that  used  168 
of  them);  and 

c)  Seven  empirical  scales  developed  by  using  the  results  of  the 
factor  analysis  to  select  the  best  items  to  reflect  each 
factor  (and  that  used  only  114  items). 

3)  Each  of  the  other  predictor  sets  was  represented  by  a  single  set 
of  scores  as  described  above  and  in  Table  3.3. 

All  results  were  adjusted  for  shrinkage  using  Rozeboom's  (1978) 
Formula  8.  Also,  results  were  computed  both  with  and  without 
correcting  for  multivariate  range  restriction.  Corrections  for 
range  restriction  were  made  using  the  9x9  intercorrelation  matrix 
among  the  ASVAB  subtests  in  the  1980  Youth  Population  (Department  of 
Defense,  1982). 

B)  Incremental  validity  results  for  each  set  of  Experimental  Battery 
predictors  over  the  ASVAB  were  computed  against  the  same  criteria 
used  in  the  preceding  step.  Once  again,  results  were  computed 
separately  by  MOS  and  then  averaged  across  the  Batch  A  MOS  and  all 
MOS  combined. 

For  this  step,  only  one  set  of  ABLE  scores  (the  seven  rational 
composites)  was  examined.  However,  incremental  validity  was 
assessed  with  respect  to  each  set  of  ASVAB  scores  described  above. 
These  results  were  adjusted  for  shrinkage  and  corrected  for 
multivariate  range  restriction.  For  the  incremental  validities 
against  the  nine  ASVAB  subtests  and  the  four  ASVAB  factors,  results 
were  also  computed  without  the  corrections  for  range  restriction. 

C)  Using  the  sample  shown  in  Table  3.4,  multiple  correlations  between 
each  set  of  predictor  scores  and  the  four  single-rater  "Peer  1" 
rating  composites,  the  four  single-rater  "Peer  2"  rating  composites, 
the  four  single-rater  supervisor  rating  composites,  and  various 
combinations  (averages)  of  these  composites  were  computed  separately 
by  MOS,  and  then  averaged  across  the  Batch  A  MOS  and  all  MOS 
combined. 

For  these  analyses,  only  one  set  of  ASVAB  scores  (the  four  ASVAB 
factor  composites)  and  one  set  of  ABLE  scores  (the  seven  rational 
composites)  were  used.  Once  again,  the  results  were  adjusted  for 
shrinkage  and  corrected  for  multivariate  range  restriction. 
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RESULTS 


Multiple  Correlations  for  ASVAB  and  LV 
Experimental  Battery  Predictors 

Multiple  correlations  for  six  predictor  sets--the  four  ASVAB  factors, 
the  single  spatial  composite,  the  eight  computer  composites,  the  three  JOB 
composites,  the  seven  ABLE  rational  composites,  and  the  eight  AVOICE 
composites— are  reported  in  Table  3.5.  As  indicated  above,  these  results  were 
computed  separately  by  MOS,  and  then  averaged  across  the  Batch  A  MOS  and 
across  all  MOS.  These  results  have  also  been  adjusted  for  shrinkage  and 
corrected  for  multivariate  range  restriction.  Results  that  have  not  been 
corrected  for  range  restriction  (but  have  been  adjusted  for  shrinkage)  are 
reported  in  Table  3.6. 

The  first  four  criteria  (i.e.,  Peer-ETS,  Peer-MPD,  Peer-PFB,  and  Peer- 
LEAO)  in  these  tables  are  the  rating  composite  scores  averaged  across  all  peer 
raters.  Likewise,  the  second  four  criteria  (Supv-ETS,  Supv-MPD,  Supv-PFB,  and 
Supv-LEAD)  are  the  rating  composite  scores  averaged  across  all  supervisor 
raters  (although,  as  noted  earlier,  the  majority  of  ratees  were  evaluated  by 
only  one  supervisor).  Finally,  the  last  three  criteria  (SK-Basic,  SK-Tech, 
and  SK-Total)  are  the  scores  from  the  School  Knowledge  tests. 

The  ASVAB  factors  were  the  single  best  set  of  predictors  for  each  of  the 
criteria  considered,  with  one  exception,  according  to  the  corrected  results 
reported  in  Table  3.5.  The  exception  was  the  average  peer  rating  of  PFB 
(which  was  better  predicted  by  ABLE).  The  pattern  of  uncorrected  results 
reported  in  Table  3.6  is  very  similar  to  that  in  Table  3.5. 

These  results  also  show  that  the  level  of  validity  of  the  ASVAB  factors 
for  predicting  the  School  Knowledge  Test  scores  was  extremely  high,  especially 
for  the  Technical  (SK-Tech)  and  Total  (SK-Total)  scores.  Likewise,  the 
spatial  composite  and  the  computer  battery  produced  high  validities  for  these 
criteria. 

Finally,  the  results  seem  to  indicate  that  peer  ratings  of  training 
performance  are  mure  highly  predicted  than  are  supervisor  ratings.  However, 
the  fact  that  the  average  peer  ratings  are  based  on  a  greater  number  of  raters 
per  ratee  than  are  the  supervisor  ratings  (and  are  therefore  probably  more 
reliable)  confounds  these  latter  findings. 

The  average  multiple  correlations  for  the  four  different  sets  of  ASVAB 
scores  are  reported  in  Tables  3.7  and  3.8  (corrected  and  uncorrected  for  range 
restriction,  respectively).  When  used  to  predict  the  average  peer  and 
supervisor  rating  composites,  the  nine  ASVAB  subtests,  the  four  ASVAB  factors, 
the  AFQT,  and  the  MOS-appropr iate  Aptitude  Area  composites  differ  only 
slightly  in  validity  (except  that  the  peer  ratings  of  Physical  Fitness,  PFB 
are  better  predicted  by  the  nine  subtests  and  the  four  factors).  However,  the 
School  Knowledge  Test  scores  are  predicted  somewhat  better  (about  three  to 
five  points)  by  the  ASVAB  subtests  and  factors  than  by  the  AFQT  or  Aptitude 
Area  composites. 
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Table  3.5 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  End-of-Training  Sample 
for  ASVA6  Factors,  Spatial,  Computer,  JOB,  ABLE  Rational  Composites,  and 
AVOICE;  Results  Corrected  for  Range  Restriction _ 


Criterion* 

MOS 

No.  Of 
MOS" 

ASVAB 

Factors 

W 

Spatial 

[1] 

Computer 

.X)B 

[3] 

ABLE 

Conp. 

[7] 

AVOICE 

[8] 

Peer-ETS 

Batch  A 

11 

41 

(07) 

35 

(05) 

36 

(05) 

24 

(06) 

19 

(09) 

22 

(07) 

All  MOS 

22 

43 

(13) 

37 

(10) 

33 

(14) 

23 

In) 

23 

(12) 

23 

(10) 

Peer-MPD 

Batch  A 

11 

25 

(04) 

22 

(05) 

21 

(05) 

09 

(07) 

19 

(05) 

11 

(07) 

All  MOS 

22 

26 

(11) 

22 

(08) 

15 

(10) 

12 

(10) 

22 

(10) 

09 

(09) 

Peer-PFB 

Batch  A 

11 

14 

(09) 

05 

(06) 

11 

(05) 

05 

(05) 

29 

(06) 

07 

(07) 

All  MOS 

22 

19 

(14) 

10 

(11) 

12 

(09) 

09 

(12) 

26 

(ID 

10 

(10) 

Peer-LEAO 

Batch  A 

11 

30 

(10) 

24 

(07) 

28 

(07) 

IB 

(09) 

22 

(09) 

i; 

(10) 

All  MOS 

22 

30 

(16) 

26 

(12) 

25 

(16) 

20 

(14) 

22 

(12) 

16 

(14) 

Supv-ETS 

Batch  A 

11 

21 

(06) 

13 

(05) 

17 

(10) 

10 

(08) 

09 

(10) 

11 

(10) 

All  MOS 

22 

27 

(15) 

22 

(ID 

18 

(13) 

10 

(10) 

11 

(12) 

10 

DO) 

Supv-MPO 

Batch  A 

11 

13 

(09) 

12 

(07) 

11 

(C8) 

06 

(06) 

05 

(06) 

06 

(06) 

Ail  MOS 

22 

16 

(16) 

14 

(11) 

10 

(13) 

06 

(08) 

05 

(07) 

C4 

(06) 

Supv-PPB 

Batch  A 

11 

11 

(07) 

09 

(05) 

09 

(08) 

06 

(05) 

11 

(09) 

07 

(07) 

All  MOS 

22 

16 

(15) 

13 

(12) 

11 

(15) 

05 

(07) 

11 

(11) 

05 

(06) 

Supv-LEAD 

Batch  A 

11 

15 

(10) 

14 

(00) 

13 

(10) 

08 

(08) 

10 

(ID 

08 

(09) 

All  MOS 

22 

19 

(17) 

1 7 

(11) 

12 

il2) 

11 

(09) 

11 

(12) 

07 

(09) 

SK-Basic 

Batch  A 

9 

58 

(06) 

57 

(06) 

57 

(06) 

38 

(05) 

30 

(07) 

37 

(03) 

All  MOS 

20 

67 

(08) 

58 

(07) 

55 

(14) 

36 

(10) 

31 

(14) 

37 

(ID 

SK-Tech 

Batch  A 

11 

76 

(05) 

53 

(05) 

61 

(05) 

41 

(07) 

33 

(05) 

44 

(07) 

All  MOS 

22 

75 

(06) 

62 

(08) 

59 

(06) 

38 

(11) 

33 

(13) 

40 

(12) 

SK-Tota 1 

Batch  A 

11 

78 

(03) 

65 

(04) 

64 

(03) 

43 

(07) 

34 

(05) 

45 

(06) 

All  MOS 

22 

77 

(05) 

65 

(07) 

62 

(07) 

40 

(11) 

35 

(14) 

42 

(13) 

Note:  Adjusted  for  shrinkage  (Roteboom  fonnula  8).  Nunt>ers  In  parentheses  are  standard  deviations. 

Hunters  in  brackets  are  the  nunters  of  predictor  scores  entering  prediction  equations.  Decimals 
omitted. 

ETS  •  Effort  and  Technical  Skill;  MPO  -  Maintaining  Personal  Oiscipllne;  PE0  -  Physical  Fitness  and 
^  Military  Bearing;  LEAD  -  Leadership  Potential;  SK  •  School  K.iowledge. 

Nurter  of  MOS  for  which  validities  were  coirputed. 
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Table  3.6 

Mean  of  Multiple  Correlations  Computed  Within>Job  for  End-of-Training  Sample 
for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE  Rational  Composites,  and 
AVOICE:  Results  lincorrected  for  Range  Restriction _ 


ASVAB  ABLE 

No.  of  Factors  Spatial  Computer  JOB  Coop.  AVOICE 

Criterion*  MOS  MOS®  [4]  [1]  _ [8] _  [3] _ [7]  _ [8] 


Peer-ETS 

Batch  A 

11 

23 

(06) 

19 

(07) 

16 

(05) 

08 

(05) 

14 

(08) 

07 

(06) 

All  MOS 

22 

23 

(08) 

20 

(08) 

14 

(07) 

07 

(07) 

12 

(09) 

04 

(06) 

Peer-MPO 

Batch  A 

11 

15 

(06) 

13 

(07) 

09 

(06) 

03 

(04) 

21 

(05) 

06 

(04) 

At 

MOS 

22 

13 

(08) 

11 

(08) 

06 

(06) 

05 

(08) 

21 

(07) 

03 

(04) 

Peer-PfS 

Batch  A 

11 

11 

(08) 

04 

(02) 

09 

(05) 

05 

(04) 

28 

(08) 

06 

(06) 

Al 

MOS 

22 

U 

(09) 

06 

(07) 

08 

(08) 

06 

(12) 

24 

(11) 

06 

(07) 

Peer  -lead 

Batch  A 

11 

14 

(08) 

12 

(04) 

14 

(04) 

07 

(06) 

18 

(03) 

09 

(06) 

All  MOS 

22 

16 

(08) 

14 

(07) 

11 

(08) 

08 

(09) 

15 

(10) 

06 

(06) 

Supv-£TS 

Batch  A 

11 

09 

(06) 

10 

(04) 

07 

(07) 

03 

(03) 

07 

(07) 

05 

(06) 

All  MOS 

22 

11 

(10) 

10 

(04) 

05 

(06) 

02 

(03) 

07 

(08) 

03 

(05) 

SupV-MPO 

Batch  A 

11 

04 

(06) 

0? 

(04) 

03 

(03) 

02 

(02) 

05 

(06) 

02 

(03) 

Al 

MOS 

22 

07 

(09) 

07 

(04) 

02 

(04) 

02 

(04) 

05 

(06) 

01 

(03) 

Supv-PfS 

Batch  A 

11 

04 

(05) 

05 

(03) 

05 

(05) 

03 

(04) 

11 

(09) 

04 

(05) 

All  MOS 

22 

06 

(07) 

07 

(06) 

04 

(07) 

02 

(05) 

09 

(10) 

02 

(04) 

Supv-LtAO 

Batch  A 

11 

07 

(06) 

07 

(04) 

06 

(05) 

04 

(04) 

09 

(09) 

04 

(06) 

All  MOS 

22 

06 

(08) 

08 

(05) 

04 

(05) 

04 

(05) 

09 

(10) 

03 

(05) 

SK-Basic 

Batch  A 

9 

47 

(11) 

36 

(12) 

29 

(09) 

16 

(08) 

10 

(09) 

17 

(11) 

All  MOS 

20 

43 

(10) 

3‘ 

(10) 

25 

(12) 

12 

(09) 

10 

(10) 

12 

(11) 

SK-Tech 

Batch  A 

11 

58 

(12) 

42 

(12) 

35 

(09) 

18 

(09) 

13 

(09) 

25 

(14) 

All  MOS 

22 

55 

(13) 

41 

(12) 

31 

(13) 

15 

(10) 

12 

(11) 

19 

(15) 

Sk-Total 

Batch  A 

11 

60 

(09) 

44 

(10) 

37 

(08) 

19 

(09) 

14 

(08) 

25 

(13) 

All  MOS 

22 

57 

(11) 

43 

(11) 

33 

(1!) 

15 

(10) 

14 

(12) 

18 

(157 

Mote:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses  are  standard  deviations. 

Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Decinuls 
omi tted. 

*  ETS  •  Effort  and  Technical  Skill;  MPO  •  Maintaining  Personal  Discipline;  PFB  •  Physical  Fitness  and 
^  Military  Bearing;  LEAD  -  Leadership  Potential;  SK  •  School  Knowledge. 

Number  of  MOS  for  which  validities  were  computed. 
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Table  3.7 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  End-of-Training  Sample 
for  ASVAB  Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  Aptitude  Area*;  Results 
Corrected  for  Range  Restriction _ 


Criterion** 

HOS 

No.  of 
M0S‘ 

ASVAB 

Subtests 

[9] 

ASVAB 

Factors 

[4] 

AFQT 

(1) 

Aptitude 

Areas* 

[13] 

Peer-ETS 

Batch  A 

11 

41 

(07) 

41 

(07) 

38 

(07) 

41 

(07) 

All  HOS 

22 

40 

(17) 

43 

(13) 

41 

(14) 

42 

(13) 

Peer-HPD 

Batch  A 

11 

25 

(05) 

25 

(04) 

24 

(06) 

25 

(05) 

All  MOS 

22 

23 

(11) 

26 

(11) 

26 

(11) 

27 

(11) 

Peer-Pf0 

Batcn  A 

11 

15 

(08) 

14 

(09) 

06 

(04) 

08 

(06) 

All  MOS 

22 

17 

(11) 

19 

(14) 

12 

(12) 

14 

(12) 

Peer-LEAD 

Batch  A 

11 

30 

(11) 

30 

(10) 

27 

(10) 

29 

(09) 

All  MOS 

22 

29 

(17) 

30 

(16) 

28 

(17) 

29 

(16) 

Supv-EIS 

Batch  A 

11 

19 

(09) 

21 

(06) 

19 

(08) 

19 

(07) 

All  MOS 

22 

24 

(16) 

27 

(15) 

24 

(15) 

25 

(16) 

Supv-MPD 

Batch  A 

11 

12 

(09) 

13 

(09) 

15 

(05) 

14 

(07) 

All  MOS 

22 

12 

(16) 

16 

(16) 

16 

(13) 

16 

(15) 

Supv-PFB 

Batch  A 

11 

09 

(09) 

11 

(07) 

10 

(05) 

10 

(05) 

All  HOS 

22 

12 

(16) 

16 

(15) 

14 

(12) 

15 

(14) 

Supv-LEAD 

Batch  A 

11 

14 

(11) 

15 

(10) 

15 

(07) 

15 

(08) 

All  MOS 

22 

17 

(16) 

19 

(17) 

19 

(14) 

20 

(15) 

SK-Basic 

Batch  A 

9 

68 

(06) 

68 

(06) 

66 

(06) 

65 

(06) 

All  MOS 

20 

68 

(07) 

67 

(08) 

65 

(12) 

64 

(09) 

SK-Tech 

Batch  A 

11 

77 

(05) 

76 

(05) 

71 

(06) 

74 

(06) 

All  MOS 

22 

75 

(06) 

75 

(06) 

68 

(12) 

71 

(10) 

SK-Total 

Batch  A 

11 

79 

(04) 

78 

(03) 

74 

104) 

76 

(04) 

All  MOS 

22 

77 

(05) 

77 

(05) 

71 

(12) 

73 

(09) 

Note;  Adjusted  for  shrinkage  (Rozeboom  formula  8). 
Numbers  In  brackets  are  the  numbers  of  predi": 
omitted. 

*  MOS-appropriate  Aptitude  Area  Composites. 

**  ETS  •  Effort  and  Technical  Skill;  MPO  •  Maintaining 

Numbers  ii  parentheses  are  standard  deviations, 
tor  scores  entering  prediction  equations.  Decimals 

Perse  lal  Discipline;  PF0  -  Physical  Fitness  and 

^  Military  Bearing;  LEAD  -  Leadership  Pctentlal;  SK  •  School  Knowledge. 
Nuuter  of  HOS  for  which  validities  were  ccnputed. 
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Table  3.8 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  End-of -Training  Sample 
for  ASVA6  Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  Aptitude  Area*:  Results 
Uncorrected  for  Range  Restriction _ 


Criterion” 

MOS 

No.  of 

HOS' 

ASVAB 

Subtests 

[9] 

ASVAB 

Factors 

[4] 

AFQT 

[1] 

Aptitude 

Areas* 

[13] 

Peer-ETS 

Batch  A 

11 

23  (05) 

23  (06) 

20  (  04) 

23  (06) 

All  MOS 

22 

20  (10) 

23  (08) 

23  (07) 

24  (08) 

Peer-MPO 

Batch  A 

11 

14  (08) 

IS  (06) 

17  (03) 

17  (05) 

All  MOS 

22 

10  (09) 

13  (08) 

16  (07) 

16  (07) 

Peer-PFB 

Batch  A 

11 

12  (07) 

11  (08) 

04  (02) 

04  (03) 

All  MO: 

22 

08  (08) 

11  (09) 

05  (05) 

05  (04) 

Peer-LEAD 

Batch  A 

11 

15  (06) 

14  (08) 

12  (04) 

15  (05) 

All  MOS 

22 

12  (08) 

16  (08) 

13  (08) 

15  (08) 

Supv-ETS 

Batch  A 

11 

06  (06) 

09  (06) 

09  (03) 

10  (03) 

All  MOS 

22 

07  (08) 

11  (10) 

11  (06) 

13  (08) 

Supv-MPO 

Batch  A 

11 

03  (05) 

04  (  06) 

C7  (03) 

07  (04) 

All  MOS 

22 

04  (06) 

07  (09) 

08  (04) 

09  (07) 

SupV-rF8 

Batch  A 

11 

04  (06) 

04  (05) 

04  (03) 

04  (03) 

All  HOS 

22 

03  (05) 

06  (07) 

C4  (05) 

06  (05) 

Supv-LEAO 

Batch  A 

11 

05  (07) 

07  (06) 

06  (03) 

08  (04) 

All  HOS 

22 

05  (06) 

06  (08) 

08  (05) 

10  (07) 

SK-0asic 

Batch  A 

9 

47  (11) 

47  (11) 

45  (10) 

42  (09) 

All  MOS 

20 

42  (11) 

43  (10) 

42  (09) 

39  (08) 

SK-Tech 

Batch  A 

11 

59  (12) 

58  (12) 

53  (08) 

54  (11) 

All  MOS 

22 

54  (17) 

55  (13) 

50  (09) 

52  (10) 

SK-Tota  1 

Batch  A 

11 

61  (09) 

60  (09) 

55  (06) 

55  (08) 

All  MC5 

22 

55  (14) 

57  (11) 

52  (09) 

52  (09) 

Note:  Adjusted  for  shrinkage  (Roteboom  formula  8).  Numbers  in  parentheses  are  standard  deviations. 

Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Decimals 
omitted. 

“  HOS-appropriate  Aptitude  Area  Composites. 

ETS  •  Effort  and  Technical  Skill;  MPO  •  Maintaining  Pe.'sonal  Discipline;  PFB  •  Physical  Fitness  and 
^  Military  Bearing;  LEAD  •  Leadership  Poterwial;  SK  -  School  Knowledge. 

Number  of  MOS  for  which  validities  were  computed. 
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The  average  multiple  correlations  for  the  three  sets  of  ABLE  scores  are 
reported  in  Tables  3.9  and  3.10  (corrected  and  uncorrected  for  range 
restriction,  respectively).  These  results  indicate  very  little  difference 
across  the  three  sets  of  scores  in  their  ability  to  predict  training 
performance. 


Table  3.9 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  End-of -Training  Sample 
for  ABLE  Rational  Composites,  A6LE-168  Composites  Hems,  and  ABLE-114 


Composites 

Items:  Results 

Corrected  for  Range 

Restriction 

Criterion* 

MOS 

No.  of 
MOS® 

able  Rational 
[7] 

ABLE -168 
[7] 

ABLE-114 

[7] 

Peer-ETS 

Batch  A 

11 

19  (09) 

19  (10) 

21  (09) 

All  MOS 

22 

23  (12) 

21  (12) 

21  (12) 

Peer -MPO 

Batch  A 

11 

19  (05) 

22  (05) 

22  (05) 

All  MOS 

22 

22  (10) 

22  (10) 

23  (10) 

Peer-OFB 

Batch  A 

n 

29  (06) 

29  (07) 

29  (07) 

All  MOS 

22 

26  (11) 

26  (11) 

26  (11) 

Peer-LEAO 

Batch  A 

11 

22  (09) 

23  (10) 

24  (09) 

All  MOS 

22 

22  (12) 

24  (11) 

24  (12) 

Supv-ETS 

Batch  A 

11 

09  (10) 

10  (10) 

10  (10) 

All  MOS 

22 

11  (12) 

12  (12) 

11  (12) 

Supv-MPO 

Batch  A 

11 

05  (06) 

06  (07) 

06  (07) 

All  MOS 

22 

05  (07) 

05  (07) 

07  (08) 

Supv-PfB 

Batch  A 

11 

11  (09) 

13  (08) 

13  (09) 

All  MOS 

22 

11  (11) 

11  (10) 

11  (10) 

Supv-LEAD 

Batch  A 

11 

10  (n) 

11  (12) 

11  (12) 

All  MOS 

22 

11  (12) 

12  (12) 

11  (12) 

SK-8asic 

Batch  A 

9 

30  (07) 

29  (08) 

31  (07) 

All  MOS 

20 

31  (14) 

29  (15) 

31  (14) 

SK-Tech 

Batch  A 

11 

33  (05) 

32  (06) 

33  (07) 

All  MOS 

22 

33  (13) 

31  (14) 

32  (13) 

SK-7ota  1 

Batch  A 

11 

3A  (05) 

34  (06) 

35  (07) 

All  MOS 

22 

35  (14) 

33  (15) 

34  (14) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses  are  standard  deviations. 

Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Decimals 
omi  tted. 

*  ETS  -  Effort  and  Technical  Skill;  MPO  -  Maintaining  Personal  Discipline:  PF6  -  Physical  Fitness  and 
Military  Bearing;  LEAD  -  Leadership  Potential;  SK  -  School  Knowledge. 

Nurber  of  MOS  for  which  validities  were  computed. 
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Table  3.10 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  End-of -Training  Sample 
for  ABLE  Rational  Composites,  ABLE-168  Conposites  Items,  and  ABLE-114 
Composites  Items;  Results  Uncorrected  for  Range  Restriction _ 


Criterion* 

MOS 

No.  of 
MOS® 

ABLE  Ratiotul 

V] 

A.RLE-168 

[7] 

ABLE-114 

[7] 

Peer-ETS 

Batch  A 

11 

14  (08) 

14  (08) 

15  (08) 

All  MOS 

22 

12  (09) 

11  (09) 

12  (09) 

Peer-MPO 

Batch  A 

11 

21  (05) 

24  (04) 

24  (05) 

All  MOS 

22 

21  (07) 

22  (08) 

23  (08) 

Peer-PFB 

Batch  A 

11 

28  (08) 

28  (10) 

28  (10) 

All  MOS 

22 

24  CD 

24  (12) 

24  (12) 

Peer-LEAD 

Batch  A 

11 

18  (08) 

19  (10) 

19  (10) 

All  MOS 

22 

15  (10) 

16  (10) 

16  (10) 

Supv-ETS 

Batch  A 

11 

07  (07) 

07  (08) 

08  (08) 

All  MOS 

22 

07  (08) 

08  (08) 

07  (08) 

Supv-MPO 

Batch  A 

11 

05  (06) 

06  (06) 

07  (06) 

All  MOS 

22 

05  (06) 

05  (07) 

06  (07) 

Supv-PFB 

Batch  A 

1  1 

11  (09) 

13  (03) 

12  (09) 

All  MOS 

22 

09  (10) 

10  (09) 

10  (10) 

Supv-LEAD 

Batch  A 

11 

09  (09) 

C9  (10) 

09  (10) 

All  MOS 

22 

09  (10) 

09  (10) 

09  (10) 

SK-8asic 

Batch  A 

9 

10  (09) 

09  (09) 

10  (05) 

All  MOS 

20 

10  (10) 

08  (10) 

09  (10) 

SK-Tech 

Batch  A 

11 

13  (09) 

14  i09) 

14  (cr.) 

All  MOS 

22 

12  (11) 

1?  (11) 

11  (C) 

SK.Tota  1 

Batch  A 

11 

14  (08) 

15  (08) 

15  (OC 

All  MOS 

22 

14  (12) 

13  (1!) 

13  (10) 

Note;  Adjusted 

for  shrinkage 

(Rozeboom  fomula  3).  Numbers  ir 

1  parentheses  are  standard  deviations. 

Nui.*ers 

omitted. 

in  Brackets  are 

the  numbers 

of  predictor  scores 

entering  prediction  equations. 

CeciTO is 

ETS  -  Effort  and  Technical  Skill;  MPD  •  Maintaining  Persona!  Discipline:  PFB  -  Physical  Fitness  and  i 

^  Military  Gearing-  LEAD  -  Leadership  Potential;  5K  -  School  Knowledge.  I 

Nunber  of  MOS  for  which  validities  were  con^juted. 


46 


Incrementa'!  Validities  for  the  Experimental 
Battery  Predictors  Over  ASVAB 


Incremental  validity  results  for  the  Experimental  Battery  predictors 
over  ASVAB  are  reported  in  Tables  3.11-3.16-  Table  3.11  contains  the  average 
multiple  correlations  of  each  set  of  Experimental  Battery  predictors  in 
combination  with  the  nine  ASVAB  subtests.  These  results  have  been  adjusted 
for  shrinkage  and  corrected  for  range  restriction.  Underlined  numbers 
indicate  validities  higher  than  those  obtained  with  the  nine  ASVAB  subtests 
alone  (which  are  reported  in  italics). 


I 

I 


The  results  in  Table  3.11  show  that  the  ABLE  composites  contribute  to 
the  prediction  of  both  the  peer  and  supervisor  average  raiings,  but  more  so 
for  peers  than  for  supervisors.  The  results  demonstrate  very  little 
incremental  validity  over  the  ASVAB  subtests  for  the  other  predictor  doinuins. 

The  average  results  for  the  incremental  validities  of  the  Experimental 
Battery  predictors  over  the  fou^  ASVAB  factors  a^'e  reported  in  Table  3.12 
(adjusted  for  shrinkage  and  corrected  for  range  restriction).  These  results 
indicate  approximately  the  same  pattern  and  degree  of  incremental  validity  of 
the  Experimental  Battery  predictors  over  ASVAB  as  were  indicated  in  Table 
3.11. 


Tables  3.13  and  3.14  report  the  average  incremental  validity  results  for 
the  Experimental  Battery  predictors  over  the  AFQT  and  the  MOS-appropriate 
Aptitude  Area  composites,  respectively.  Also  adjusted  for  shrinkage  and 
corrected  for  range  restriction,  the  results  in  these  tables  indicate  greater 
incremental  validities  over  ASVAB  than  those  shown  in  Tables  3.11  end  3.12. 
This  is  attributable,  however,  to  the  lower  levels  o^  validity  associated  with 
the  AFQT  and  Aptitude  Area  composites  for  certain  criteria  (e.g.  ,  the  School 
Knowledge  Test  scores),  rather  than  to  greater  levels  of  validity  from  tne 
combined  use  of  these  ASVAB  scores  and  the  Experimental  Battery  predictors. 

The  final  incremental  validity  results  are  reported  in  Tables  3.15  and 
3.16.  These  tables  contain  the  average  incremental  validity  results  for  the 
Experimental  Battery  predictors  over  the  nine  ASVAB  subtests  and  the  four 
ASVAB  factors,  respectively,  uncorrected  for  range  restriction.  The  primary 
reason  for  computing  these  results  was  to  compare  them  with  the  corrected 
results  reported  in  Tables  3.11  and  3.12  to  determine  how  the  estimates  ov 
incremental  validity  are  affected  by  the  range  restriction  corrections. 

These  comparisons  indicate  greater  estimated  levels  of  incremental 
validity  for  the  Experimental  Battery  when  the  data  are  not  corrected  for 
range  restriction.  Specifically,  when  there  is  no  correction  for  range 
restriction,  the  incremental  validities  obtained  for  the  ABLE  are  mostly  two 
to  three  points  higher  than  the  incremental  validities  obtained  after 
correcting  for  range  restriction,  with  the  difference  being  as  mucn  as  five 
points  in  some  cases.  For  the  other  Experimental  Battery  predictors,  the 
differences  between  the  incremental  validities  estimated  on  corrected  and 
uncorrected  data  are  more  modest.  The  incremental  validities  estimated 
without  the  range  restriction  correction  are  only  one  point  higher  (if  at  all) 
than  the  corrected  incremental  validities. 
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Table  3.11 


Mean  of  Incremental  Correlations  Over  ASVAB  Subtests  Computed  Within-Job  for 
End-of-Tralning  Sample  for  Spatial,  Computer,  JOB.  ABLE  Composites,  and 
AVOICE:  Results  Corrected  for  Range  Restriction _ 


Criterion* 

MOS 

No.  of 
NOS'’ 

A9 

ASVAB 

Subtests 

[9] 

A9* 

Spatial 

[101 

A9r 

Conputcr 

[17] 

A9» 

JOB 

[3] 

A9* 

ABLE 

Comp. 

[7] 

A9» 

AVOICE 

[8] 

oeer-ETS 

Batch  A 

n 

41  (07) 

42  (07) 

41  (Ob) 

41  (07) 

44  (06) 

41  (07) 

All  MOS 

22 

40  (17) 

41  (17) 

39  (16) 

40  (16) 

4?  (15) 

39  (16) 

Pcer-MPO 

Batch  A 

1! 

2S  (05) 

25  (05) 

23  (08) 

24  (06) 

34  1,05) 

23  (08) 

All  MOS 

22 

23  (11) 

22  (12) 

20  (12) 

21  (12) 

3?  (10) 

20  (11) 

Peer-PF  0 

Batch  A 

11 

15  (03) 

15  (08) 

17  (07) 

15  (09) 

31  (09) 

16  (09) 

All  MOS 

22 

12  (11) 

16  (11) 

TI  (12) 

18  (lb) 

W  (13) 

Tb  (11) 

Peer-LEAD 

Batch  A 

11 

30  (11) 

29  l'.\) 

31  (09) 

29  (13) 

35  (OS) 

30  (13) 

All  MOS 

22 

29  (17) 

29  (17) 

I7  (19) 

29  (18) 

31  (lb) 

27  (19) 

Supv-ETS 

Batch  -A 

11 

19  (09) 

19  (09) 

16  (11) 

18  (09) 

17  (13) 

16  (13) 

All  MOS 

7  ? 

24  (16) 

24  (16) 

21  (15) 

22  (!6; 

24  (20) 

20  (16) 

S'jpv-MPO 

Batch  A 

11 

12  (09) 

12  (09) 

10  (09) 

10  (10) 

12  (11) 

10  (10) 

Al  1  MOS 

2? 

13  (16) 

13  (15) 

10  (15) 

12  (16) 

13  (16) 

10  (14) 

Supy-PFO 

Patch  A 

11 

03  (09) 

09  (09) 

09  (09) 

13  (11) 

09  (10) 

hll  MOS 

22 

12  (16) 

12  (15) 

(12) 

11  (14) 

H  (15) 

09  (12) 

Supv-LEAO 

Batch  A 

11 

19  (11) 

14  (11) 

13  (11) 

13  (12) 

(14) 

13  (12) 

All  MOS 

22 

i'  (16) 

17  (16) 

13  (14) 

17  (16) 

VS  (17) 

14  (14) 

SK-3asic 

Batch  A 

9 

68  (06) 

69  (06) 

68  (06) 

68  (06) 

68  (061 

68  (06) 

Al  1  MOS 

20 

68  (07) 

(C7) 

65  (14) 

67  (08) 

68  (03) 

66  (11) 

Sh-Tech 

Batch  A 

11 

vV  (05) 

27  (05) 

77  (05) 

77  (05) 

77  (05) 

77  (06) 

All  MOS 

22 

25  (06) 

75  (06) 

75  (06) 

75  (06) 

75  (07) 

74  (08) 

SK-Tora  1 

Batch  A 

:i 

29  (OA) 

79  (03) 

79  (03) 

78  (04) 

79  (04) 

.^8  (04) 

Ail  MOS 

22 

-V  (05) 

77  (05) 

77  (05) 

77  i05) 

77  (C5) 

76  (07  ) 

Note.  Adjusted  for  shrinkage  (Rozebooiti  for(i>j)a  8).  Nunt>ers  in  parentheses  are  standard  oeviations. 

hurrtiers  in  brackets  are  the  numbers  of  predictor  scores  entering  prediction  equations.  Multiple  Rs 
for  ASVAB  Subtests  alone  are  in  italics.  Unoerltnea  numbers  denote  ixiltiple  Rs  greater  than  for 
ASVA8  Subiests  alone.  Decinvils  omitted. 

“  uTS  •  Effort  and  Technical  Skill;  MPO  -  Maintaining  Persorn!  D'scip'ine;  PPB  -  Physica’  f'tness  and 
p  Military  Bearing;  tEAD  -  Leadership  Potential;  SK  •  Scnool  Xrio« ledge. 

Iluifber  MOS  for  «hich  validities  were  computed. 
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ble  3.12 


•n  of  Zncremantal  Correlations  Over  ASVAB  Factor's  Computed  Within-Job  for 
d-of-Training  Sample  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 
OICE:  Results  Corrected  for  Range  Restriction _ 


M  A4* 


terlon* 

HOS 

ho.  of 
nos® 

ASVAB 

Factors 

[4] 

A4< 

Spatial 

[5] 

A4* 

Conputer 

[12] 

A4<- 

xe 

[7J 

ABLE 

Cotrp. 

[11] 

A4* 

AVOICF. 

[12] 

r-ETi 

Eatch  A 

11 

Al 

(07] 

42  (07) 

t'  (06) 

41  (07) 

M  (06) 

41  (07) 

All  HOS 

22 

A3 

03] 

17  (14) 

4.  (16) 

42  (13) 

I?  UD 

41  (14) 

r-MPC 

Batch 

11 

25 

(04] 

25  (05) 

24  (05) 

25  (05) 

34  (00) 

24  (07) 

All  1 

22 

26 

(11) 

25  (11) 

22  (12) 

25  (12) 

n  (11) 

22  (11) 

-.PfB 

Batch  A 

1 1 

lA 

(09) 

13  (09) 

ii  (07) 

15  (09) 

31  (09) 

15  (09) 

All  nos 

22 

19 

(14) 

16  (14) 

T5  (12) 

B  (17) 

^  (14) 

15  (11) 

r-LtU) 

Batch  A 

11 

30 

(10) 

30  (10) 

31  (08) 

30  (11) 

35  (09) 

29  (13) 

All  nos 

22 

30 

(16) 

30  (17) 

76  (18) 

li  08) 

M  (15) 

28  (18) 

'-[IS 

Batch  A 

11 

21 

(06) 

21  (07) 

19  (09) 

20  (06) 

19  (12) 

17  (12) 

All  nos 

22 

22 

(15) 

26  (15) 

24  (15) 

25  (15) 

25  (19) 

22  (15) 

V  MPi) 

Batch  A 

11 

13 

(09) 

12  (03) 

11  (09) 

11  (09) 

13  (U) 

11  (10) 

A')  nos 

22 

16 

(16) 

16  (16) 

12  (17) 

14  (17) 

16  (16) 

11  (14) 

/■I'li) 

Batch  A 

11 

11 

(0/' 

11  (07) 

10  (08) 

10  (07) 

16  (09) 

10  (09) 

All  nos 

22 

16 

(Ih) 

15  (14) 

12  (15) 

14  (13) 

S  (13) 

11  (13) 

V  *LE  'I' 

Batch  A 

11 

15 

(lOi 

14  (10) 

14  (11) 

14  (10) 

(13) 

13  (12) 

*  1 1  nuS 

'll 

IV 

'If) 

IV  (i/) 

1‘.  (15) 

19  116) 

Z5  (in 

15  115) 

Sat  ic 

Batch  A 

9 

63 

(06) 

69  (06) 

68  (OG) 

60  (06) 

68  (07) 

68  (06) 

All  f«b 

20 

62 

(08) 

S  (03) 

05  (IG) 

67  (09) 

66  (  ’1) 

66  (10; 

'i;n 

Bitch  A 

11 

2( 

(OS) 

77  (05) 

71!  (05) 

76  (05) 

76  (05) 

76  (05) 

All  nos 

a 

25 

(06) 

T5  (06) 

75  (05) 

75  (06) 

75  (07) 

74  (07) 

lota  1 

Ba'ch  A 

11 

28 

(03) 

79  (03) 

79  (03) 

78  (03) 

79  (03) 

78  (04) 

Ai;  nos 

11 

22 

(OS) 

77  (05) 

77  (C5) 

r  rs) 

77  (061 

76  (06) 

!.  A'J  1 /i'to  fo''  ihi  irikag*  for  <ild  fl).  HumL-ers  in  parentneses  are  standard  deviafloni, 

Nunjrfiri  In  [iijctcts  are  the  nu,Tti«"  s  of  preatclor  scores  irntering  prediriton  equattons.  Multiple  Ri 
tor  (a'.ioii  a  .one  arc  m  ita'i  s.  Urtdei  I  Imed  nonfctrs  denote  nvUlple  Rs  greater  man  for 

/'jVAii  lectom  alonv'.  Oes.rviis  offi’i'..  i. 

I*. 'for',  and  lijirnlcal  Skill:  ri'O  .  Mairuainirj  Personal  Discipline-  PIB  •  Physical  ritness  and 
■  141/  6*a'irvj:  iCVI  *  ^eldership  Potent  lal;  $K  •  School  K.ix/wU  iqo. 
ef.i,'  of  MO'j  foi  whi'.i.  .jilditiev  w^re  tonpule'I. 
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Table  3.13 


Mean  of  Incremental  Correlatioiis  Over  AFQT  Computed  Within-Job  for  End-of 
Training  Sample  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE: 


Results  Corrected  for  Range  Restriction 


Criterion* 

MOS 

No.  of 
MOS* 

AFQT 

[1] 

M 

AfQT* 

Coeputer 

[9) 

AFQT* 

JOB 

[4] 

AFQT* 

ABLE  Comp. 
[8] 

AFQT* 

AVOICE 

[9] 

Peer-ETS 

Batch  A 

11 

ja  (07) 

40  (06) 

39  (06) 

38  (  07) 

41  (06) 

37  (08) 

All  MOS 

22 

42  (14) 

TT  (14) 

15  (16) 

38  (16) 

31  (12) 

38  (14) 

Peer-MPO 

Batch  A 

11 

24  (06) 

25  (05) 

23  (06) 

22  (09) 

33  (08) 

22  (10) 

All  MOS 

22 

26  (11) 

Ta  (13) 

21  (12) 

24  (12) 

(12) 

20  (14) 

Peer-PFB 

Batch  A 

11 

06  (04) 

04  (05) 

13  (03) 

05  (06) 

29  (06) 

08  (06) 

All  HOS 

22 

12  (12) 

10  (12) 

14  (12) 

10  (14) 

S  (12) 

17  (12) 

Peer. LEAD 

Batch  A 

11 

27  (10) 

27  (10) 

29  (09) 

26  (11) 

32  (12) 

26  (13) 

All  HOS 

22 

28  (17) 

28  (17) 

77  (18) 

27  (18) 

S  (17) 

24  (20) 

Supv-ETS 

Batch  A 

11 

19  (08) 

20  (08) 

19  (10) 

17  (09) 

17  (12) 

15  (13) 

All  MOS 

22 

24  (15) 

7?  (15) 

20  (14) 

21  (15) 

22  (15) 

19  (14) 

Supv-MPO 

Batch  A 

11 

15  (05) 

14  (07) 

11  (08) 

13  (07) 

13  (10) 

10  (09) 

All  MOS 

22 

16  (13) 

12  (14) 

11  (14) 

12  (14) 

11  (12) 

09  (11) 

Supv-PFB 

Batcn  A 

11 

10  (05) 

09  (06) 

09  (07) 

09  (04) 

15  (08) 

08  (07) 

All  MOS 

22 

14  (12) 

12  (13) 

12  (14) 

10  (10) 

T4  (11) 

08  (10) 

Supv-LEAD 

Batch  A 

11 

IS  (07) 

14  (08) 

13  (10) 

14  (08) 

15  (12) 

12  (10) 

All  MOS 

22 

19  (14) 

16  (14) 

13  (12) 

17  (13) 

17  (13) 

12  (11) 

SK-Bas 1C 

Batch  A 

9 

66  (06) 

67  (06) 

66  (06) 

66  (06) 

66  (06) 

66  (06) 

All  HOS 

20 

6S  (12) 

57  (09) 

64  (16) 

63  tl6) 

63  (16) 

63  (16) 

SK-Tecn 

Botch  A 

11 

71  (06) 

73  (05) 

72  (06) 

71  (06) 

71  (06) 

73  (06) 

All  HOS 

22 

68  (12) 

75  (12) 

W  (10) 

67  (16) 

67  (16) 

55  (16) 

SK-Total 

Batch  A 

U 

74  (04) 

76  (03) 

75  (04) 

74  (04) 

74  (05) 

75  (04) 

All  MOS 

22 

21  (12) 

7±  (10) 

^  (11) 

70  (16) 

71  (16) 

71  (16) 

Hof":  Adjusted  for  Shrinkage  (Ro^eboon  fonTwIa  8).  NuAt)ers  in  parentheses  are  Standard  deviations. 

Hiiiwcrj  in  brackets  are  the  nutrtiers  of  predictor  s  ;ores  entering  prediction  equations.  Correlations 
for  AFQT  alone  are  in  italics,  underlined  nuitfters  denote  owitiple  Rs  greater  than  for  AFQT  alone. 
Decimals  onitted. 

*  FTS  •  Cffor:  and  technical  Skill;  MPh  .  Maintaining  Personal  Oiscipllne;  PF0  -  •'hysical  Fitness  and 
^  Military  Bearing;  LEAD  -  Leadership  Potential;  SK  •  School  Knowledge. 

Hunter  of  f'Uj  for  ..nich  validities  .^re  ccunputed. 


Table  3.14 


Mean  of  Incremental  Correlations  Over  ASVAB  HOS-Appropriate  Aptitude  Area 
Composites  Computed  Within-Job  for  End-of-Training  Sample  for  Spatial, 
Computer,  JOB,  ABLE  Composites,  and  AVOICE:  Results  Corrected  for  Range 


Restriction 


M.* 

AA* 

AA^ 

AA* 

AA+ 

No.  of 

AA‘ 

Spatial 

Coflputer 

JOB 

ABLE  Coup. 

AVOICE 

Criterion* 

MOS 

MOS" 

[1] 

[7] 

[9] 

[4] 

[8] 

[9] 

Peer-ETS 

Batch  A 

11 

40  (07) 

41  (07) 

41  (06) 

40  (07) 

43  (06) 

40  (07) 

AH  MOS 

22 

4^  (13) 

TT  (14) 

35  he) 

40  (14) 

43  (12) 

40  (14) 

Peer-HPD 

Batch  A 

11 

2S  (OS) 

25  (06) 

25  (06) 

24  (08) 

33  (07) 

2‘  (07) 

All  MOS 

22 

26  (11) 

24  (13) 

21  (12) 

25  (11) 

H  (11) 

22  (14) 

Peer-PFB 

Batch  A 

11 

08  (06) 

05  (07) 

14  (03) 

08  (07) 

29  (07) 

09  (06) 

All  MOS 

22 

13  (12) 

11  (12) 

H  (10) 

12  (14) 

75  (12) 

IT  (11) 

Peer-LEAD 

Batch  A 

11 

29  (09) 

28  (10) 

31  (08) 

28  (11) 

34  (09) 

28  (13) 

All  MOS 

22 

29  (16) 

28  (16) 

7?  (17) 

29  (18) 

M  (16) 

26  (19) 

Supv-ETS 

Batch  A 

11 

20  (06) 

18  (08) 

19  (10) 

18  (09) 

18  (1>) 

16  (12) 

All  MOS 

22 

^5  (16) 

24  (16) 

21  ('5) 

23  (16) 

23  (16) 

21  (15) 

Supv-HPO 

Batch  A 

11 

14  (07) 

12  (08) 

11  (09) 

12  (08) 

12  (11) 

10  (10) 

All  MOS 

22 

16  (15) 

13  (15) 

12  (14) 

13  (16) 

12  (14) 

10  (13) 

Supv-PFB 

Batch  A 

11 

:o  :05) 

07  (05) 

05  (07) 

08  (05) 

14  (09) 

08  (O'/) 

All  MOS 

22 

13  ilA) 

12  (14) 

12  (14) 

11  (12) 

H  (11) 

09  (10) 

Supv-LEAO 

Batch  A 

11 

IS  (08) 

14  (09) 

13  (11) 

14  (09) 

16  (13) 

13  (11) 

All  MO: 

22 

20  (15) 

10  (15) 

14  fl4) 

18  (15) 

T5  (14) 

13  (13) 

SK-Bas ic 

Batch  A 

9 

65  (06) 

66  (06) 

66  (  06) 

65  (06) 

65  (06) 

65  (06) 

All  MO;'. 

20 

64  (09) 

55  (08) 

57  (16) 

62  (12) 

61  (16) 

63  (12) 

SK-Tech 

Batcii  A 
All  MOS 

11 

22 

23  (OC) 

71  (10) 

74  (05) 

>1  (11) 

74  (06) 

n  (08) 

74  f06) 

70  (»2) 

74  (06) 

70  (17) 

74  (06) 

59  (16) 

SK-lota  1 

Batch  * 

U 

75  (04) 

76  (04) 

76  (04) 

76  (04) 

76  (04^ 

76  (04' 

All  MOS 

22 

73  (09) 

75  (09) 

2  (08) 

73  (11) 

77  (17) 

77  (14) 

Note:  Adjuster)  fc'  s.’rinkage  (Rozeboom  fortiwU  8).  N'inOers  tn  parentheses  are  standard  deviations. 

Nunfcers  ir  brackets  are  the  ntutOers  oh  predictor  scores  entering  p  edictlon  equations.  Correlations 
for  MOS-approprlate  Aptitude  Area  co.tpositcs  alone  arc  in  Italic.’;.  Underlined  nunters  denote 
iTultiple  Rs  greater  than  for  Aptitude  Area  cotrposUes  alone.  Oedrals  omitted. 

*  '!>  -  Effort  and  technical  Skill;  MPO  -  Maintaining  Personal  Discipline;  •  Physical  f  Itnoi-s  and 
'llittary  Bearl.ng;  LEAD  -  Leadersnlp  Potential;  SK  -  School  Knowledge. 

Hurfcer  ct  MOS  for  which  validities  were  coirputed. 

‘  .'A  •  ASVAB  lOS-appropr late  Aptitude  Area  comoosUes. 
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Table  3.15 

Hean  of  Incremental  Correlations  Over  ASVAB  Subtests  Crmputed  W1th1n~Job  for 
End-of-Training  Sample  f<ir  Spatial,  Cosgiuter,  JOB,  ABLE  u'^mposltes,  and 


AVOICE;  Results  Uncorrected  for  Range  Restriction 


Criterion* 

HOS 

No.  of 
HOS* 

A9 

ASVAB 

Suhtests 

[9] 

M* 

Spatial 

[10] 

A9* 

Coa^uter 

[17] 

A9* 

JOB 

[12] 

A9+ 

ABLE  Conp. 
[16] 

A9* 

AVOICE 

[17] 

Pe«r-ETS 

Batch  A 

11 

23  (OS) 

24  (05) 

23  (07) 

22  (07) 

27  (06) 

21  (08) 

All  MCS 

22 

20  (10) 

S  (10) 

18  (11) 

19  (10) 

U  (12) 

18  (11) 

Peer-MPO 

Batch  A 

11 

14  (08) 

14  (08) 

12  (09) 

14  (08) 

28  (04) 

13  {  j8) 

All  MOS 

22 

10  (09) 

10  (09) 

08  (09) 

09  (09) 

n  (11) 

08  k09) 

Peer-PFB 

Batch  A 

11 

12  (07) 

11  (07) 

12  (07) 

11  (O'" 

28  (11) 

12  (09) 

All  MOS 

22 

08  (08) 

07  (08) 

09  (10) 

10  (1m 

H  (13) 

08  (09) 

Peer-LEAO 

Batch  A 

11 

15  (06) 

15  (06) 

16  (07) 

16  (07) 

24  (07) 

17  (08) 

AM  MOS 

22 

12  (08) 

13  (09) 

11  (10) 

n  (09) 

T9  (10) 

17  (09) 

Supv-ETS 

Batch  A 

11 

06  (06) 

07  (06) 

07  (08) 

06  (07) 

08  (09) 

06  (08) 

All  MOS 

22 

O;  (08) 

®  (08) 

(J?  (07) 

06  (08) 

H  (12) 

05  (07) 

Supv-KPO 

Batch  A 

U 

03  (05) 

03  (05) 

02  (04) 

03  (04) 

06  (07) 

03  (05) 

All  MOS 

22 

04  (06) 

03  (05) 

02  (04) 

03  (05) 

W  (07) 

02  (04) 

Supv-PFB 

Batch  A 

1  1 

04  (06) 

04  (06) 

04  (05) 

04  (06) 

09  (10) 

OS  (07) 

All  MOS 

22 

03  (06) 

03  (05) 

02  (04) 

03  (06) 

M  (10) 

ZJ  (06) 

Supv-LEAD 

Batch  A 

11 

05  (07) 

05  (97) 

05  (07) 

06  (07) 

10  (11) 

06  (09) 

All  MOS 

22 

05  (06) 

04  (06) 

03  (06) 

M  (06) 

W  (10) 

M  (07) 

SK-Basic 

Batch  A 

9 

42  (11) 

48  (ID 

47  (11) 

47  (11) 

47  (11) 

46  (11) 

All  HOS 

20 

42  {U} 

38  (18) 

40  (14) 

42  (13) 

38  (16) 

SK-Tech 

Batch  A 

11 

59  (12) 

59  (12) 

59  (12) 

59  (12) 

59  (11) 

59  (12) 

All  MOS 

22 

5f  (17) 

54  (17) 

54  (16) 

54  (15) 

54  (14) 

52  (18) 

SK-Total 

Batch  A 

11 

01  (09) 

6'  (09) 

61  (09) 

60  (09) 

61  (09) 

60  (09) 

All  MOS 

22 

5i  (14) 

55  (15) 

55  (16) 

55  (13) 

^  (12) 

53  (17) 

Mote:  A(l,usted  for  shrinkage  (Rotebooni  for-xjla  9).  f.u/iibers  in  parentheses  are  standard  deviations. 

Ml  tiers  in  brackets  are  the  nuneers  of  predlcfjr  scores  entering  prediction  eguatioiis.  Multiple  Rs 
Mr  ASVAB  Subtests  alone  are  In  italics.  Underlined  numbers  denote  nultiple  Rs  greater  than  for 
ASVAB  Subtests  alone.  Decimals  omitted. 

*  £TS  •  tffort  and  Technical  Skill;  MPO  -  Maintaining  Personal  Discipline;  PFB  •  Physical  Fitness  and 
Military  Bearing;  LtAD  •  Leadership  Potential;  SK  •  School  Knowledge, 
f'umoer  of  KOS  for  which  validities  were  computed. 
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Table  3.16 

Hean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within'-Job  for 
End-of-Training  Sample  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 


AVOICE;  Results  Corrected  for  Range  Restriction 


Criterion* 

H05 

No.  of 
MOS* 

A4 

ASVAB 

Factors 

[4] 

M* 

Spatial 

f5] 

A4* 

Computer 

[12] 

A4* 

JOB 

[7] 

A4* 

ABLE 

Conp. 

[11] 

A4* 

AVOICE 

[12] 

Peer-ETS 

Batch  A 

11 

(05) 

24 

(06) 

23  (07) 

23  (07) 

27 

(09) 

22  (07) 

All  MOS 

22 

(08) 

53 

(10) 

19  (11) 

22  (09) 

H 

(10) 

19  (09) 

Peer-MPO 

Batch  A 

11 

IS  (06) 

15 

(06) 

13  (08) 

15  (06) 

28 

(04) 

13  (08) 

All  MOS 

22 

U  (08) 

13 

(09) 

09  (09) 

13  (09) 

(09) 

09  (09) 

Peer-PFB 

Batch  A 

11 

II  (08) 

10 

(08) 

13  (07) 

11  (09) 

29 

(11) 

12  (09) 

All  MOS 

22 

II  (09) 

10 

(09) 

TS  (10) 

12  (14) 

H 

(12) 

10  (09) 

Peer -LEAD 

Batch  A 

11 

M  (08) 

14 

(08) 

16  (07) 

15  (08) 

23 

(09) 

17  (08) 

All  MOS 

22 

16  (08) 

15 

(09) 

T7  DO) 

T5  (10) 

IT 

(09) 

TT  (10) 

Supv-ETS 

Batch  A 

11 

09  (06) 

09 

(06) 

07  (08) 

08  (06) 

10 

(09) 

08  (07) 

All  MOS 

22 

II  (10) 

11 

(09) 

07  (08) 

09  (08) 

n 

In) 

07  (07) 

Supv-MPO 

Batch  A 

11 

04  (06) 

04 

(06) 

04  (04) 

04  (05) 

0 

l08) 

04  (05) 

All  MOS 

22 

07  (09) 

07 

(09) 

05  (10) 

05  (08) 

77 

(08) 

03  (04) 

Supv-PFB 

Batch  A 

11 

04  (05) 

04 

(05) 

05  (05) 

03  (04) 

12 

(0?) 

05  (07) 

All  MOS 

22 

06  (07) 

06 

(06) 

(04) 

04  (06) 

15 

(10) 

51  (06) 

Supv-LEAO 

Batch  A 

11 

07  (06) 

07 

(06) 

06  (07) 

07  (06) 

11 

(10) 

07  (08) 

All  MOS 

22 

06  (08) 

06 

(08) 

04  (06) 

07  (06) 

ii 

(10) 

05  (06) 

SK-Bas1c 

Batch  A 

9 

47  (11) 

48 

(11) 

47  (11) 

47  (11) 

47 

(11) 

46  (11) 

All  MOS 

20 

43  (11) 

(ID 

40  (17) 

42  (11) 

43 

(12) 

39  (15) 

SK-Tech 

Batch  A 

11 

58  (12) 

59 

(12) 

59  (12) 

58  (12) 

59 

(11) 

56  (12) 

All  MOS 

22 

55  (13) 

5£ 

(14) 

T?  (14) 

65  (13) 

(12) 

53  (18) 

SK-Total 

Batch  A 

11 

60  (09) 

61 

(09) 

61  (09) 

60  (09) 

61 

(08) 

60  (09) 

All  MOS 

22 

5/  (11) 

S7 

(12) 

I?  (13) 

56  (IM 

57 

(11) 

5S  (14) 

Note:  Adjusted  for  shrinkage  (Roieboom  fonnila  8).  Numbers  in  parentheses  are  srandard  deviations. 

Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering  prediction  eguatlons.  Multiple  Rs 
for  ASVAB  Factors  alone  are  in  italics.  Underlined  numbers  denote  nultiple  Rs  greater  than  for 
ASVAB  Factors  alone.  Decimals  omitted. 

*  E7S  •  Effort  and  Technical  Skill;  MPO  -  Maintaining  Personal  Discipline;  PFB  -  “hysical  Fitness  and 
^  Military  Bearing:  LEAD  •  Leadership  Potential;  SK  •  School  Knowledge. 

Number  of  KOS  for  which  validities  were  computed. 
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Comparison  of  Multiple  Correlations  for  Predicting 
Peer  vs.  Supervisor  Rating  Composites 


Results  of  analyses  designed  to  compare  the  validities  of  the  ASVAB  and 
Experimental  Battery  scores  as  predictors  of  peer  and  supervisor  ratings  are 
contained  in  Table  3.17.  Recall  that  differences  between  validities 
associated  with  peer  and  supervisor  ratings  reported  until  now  have  been 
confounded  with  differences  in  the  average  number  of  raters  per  ratee  for  the 
peer  and  supervisor  ratings.  In  Table  3.17,  however,  average  multiple 
correlations  (adjusted  for  shrinkage  and  corrected  for  range  restriction)  for 
the  ASVAB  and  Experimental  Battery  predictors  are  reported  for  the  two  sets  of 
single-rater  peer  rating  composites  (labeled  in  the  table  as  PI  and  P2)  and 
the  one  set  of  single-rater  supervisor  rating  composites  (SI). 

In  prior  analyses,  trainees  in  MOS  13B  had  been  split  into  two  groups 
("self-propelled"  and  "towed")  due  to  differences  in  the  School  Knowledge 
Tests  that  each  was  administered,  depending  on  the  particular  weapon  with 
which  they  were  trained.  Because  the  School  Knowledge  scores  were  not 
required  for  the  present  comparisons,  the  dichotomy  was  dropped.  Therefore, 
the  averages  in  Table  3.17  are  based  on  one  fewer  MOS  than  those  in  earlier 
tables. 

Table  3.17  indicates  that  the  results  for  the  Peer  1  (PI)  and  Peer  2 
(P2)  rating  composites  are  approximately  the  same.  This  is  not  surprising 
considering  that  both  are  sets  of  single-rater  peer  ratings  of  exactly  the 
same  set  of  ratees.  On  the  other  hand,  the  results  indicate  that  the 
validities  of  the  ASVAB  and  Experimental  Battery  scores  are  often  much  lower 
when  used  to  predict  the  single-rater  supervisor  ratings  (SI)  than  when  they 
are  used  to  predict  either  of  the  single-rater  peer  ratings. 

Validity  results  for  three  different  combinations  of  the  single-rater 
composites  are  reported  in  Table  3.18.  These  combinations  are:  two  sets  of 
two-rater  rating  composites  (one  composed  of  the  average  of  the  PI  and  SI 
ratings  [P151],  the  other  composed  of  the  average  of  the  PI  and  P2  ratings 
[P1P2J)  and  one  set  of  three-rater  composites  composed  of  the  average  of  the 
PI,  P2,  and  SI  ratings  (PPS). 

These  results  indicate  that  the  validities  associated  with  the  average 
of  the  two  single-rater  peer  rating  composites  tend  to  be  higher  than  those 
associated  with  the  average  of  the  single-rater  peer  and  single-rater 
supervisor  rating  composites.  Furthermore,  the  validities  associated  with  the 
average  of  the  three-rater  (two  peers  and  one  supervisor)  composites  are 
generally  no  greater  than  the  validities  associated  with  the  average  of  the 
two  peer  ratings  alone. 


SUMMARY  AND  FUTURE  ANALYSES 

Given  the  results  from  the  validity  analyses  just  presented,  the 
following  conclusions  appear  justified: 

•  The  ASVAB  has  a  substantial  correlation  with  all  of  the  end-of- 
training  performance  components. 
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Table  3.17 


Hean  of  Multiple  Correlations  Computed  Within-Job  for  ASVAB  Factors,  Spatial, 
Computer,  JOB,  ABLE  Composites,  and  AVOICE  as  Predictors  of  Single-Rater  Peer 
and  Supervisor  End-of-Training  Ratings _ 


No.  of 


Criterion* 

HOS 

HOS' 

Pl-ETS 

Batch  A 

10 

All  MOS 

21 

Pl-MPO 

Batch  A 

10 

All  MOS 

21 

Pl-PFB 

Batch  A 

10 

All  MOS 

21 

PI-LEAD 

Batch  A 

10 

All  MOS 

21 

P2-ETS 

Batch  A 

10 

All  MOS 

21 

P2-MP0 

Batch  A 

10 

All  MOS 

21 

P2.PFB 

Batch  A 

10 

All  MOS 

21 

P2.LEAD 

Batch  A 

10 

All  MOS 

21 

Sl-ETS 

Batch  A 

10 

All  MOS 

21 

Sl-MPO 

Batch  A 

10 

All  MOS 

21 

Sl-PFB 

Batch  A 

10 

All  MOS 

21 

SI-LEAD 

Batch  A 

10 

All  MOS 

21 

ASVAB 

Factors  Spatial  Conputer 

[4]  [1]  [8] 


30 

(08) 

25 

(05) 

23 

(10) 

30 

(14) 

27 

Ul) 

23 

(14) 

16 

(09) 

14 

(06) 

11 

(07) 

18 

(IS) 

16 

(ID 

10 

(09) 

12 

(08) 

05 

(05) 

08 

(08) 

16 

US) 

11 

(12) 

11 

0.5) 

22 

(11) 

18 

(08) 

17 

(11) 

24 

(12) 

20 

(11) 

16 

(13) 

30 

(04) 

25 

(05) 

25 

(06) 

34 

(14) 

28 

(10) 

26 

(09) 

17 

(03) 

15 

(03) 

12 

(07) 

20 

(13) 

19 

(08) 

11 

(09) 

12 

(07) 

05 

(06) 

07 

(07) 

15 

(13) 

10 

f09) 

07 

(08) 

21 

(09) 

17 

(07) 

18 

(11) 

23 

(14) 

22 

(11) 

19 

(14) 

19 

(07) 

17 

(05) 

16 

(10) 

25 

(17) 

22 

(13) 

17 

(14) 

12 

(06) 

11 

(06) 

10 

(06) 

15 

(16) 

13 

(D) 

10 

(13) 

09 

(08) 

08 

(04) 

07 

(07) 

14 

(15) 

13 

(12) 

10 

(14) 

13 

(09) 

13 

(07) 

11 

(09) 

18 

(17) 

17 

(11) 

10 

(11) 

ABLE 

JOB  Conp.  AVOICE 

[3]  [7]  [8] 


15 

14 

(05) 

(10) 

11 

14 

(08) 

(13) 

16 

13 

(08) 

(12) 

05 

(05) 

12 

(06) 

05 

(06) 

06 

(10) 

18 

(13) 

08 

(12) 

02 

(04) 

17 

(09) 

07 

(07) 

04 

(07) 

IS 

D6) 

06 

(08) 

12 

(09) 

15 

(07) 

12 

(10) 

11 

(09) 

14 

(13) 

09 

(11) 

17 

(04) 

13 

(09) 

15 

(08) 

17 

(10) 

17 

(12) 

19 

(18) 

07 

(05) 

13 

(08) 

08 

(06) 

09 

(10) 

14 

(10) 

07 

(08) 

03 

(05) 

18 

(10) 

06 

(05) 

03 

(06) 

16 

(12) 

07 

(11) 

12 

(08) 

13 

(11) 

12 

(07) 

16 

(13) 

13 

(11) 

13 

(10) 

10 

(08) 

09 

UO) 

11 

(10) 

11 

(11) 

11 

(13) 

11 

(10) 

05 

(05) 

06 

(06) 

06 

(05) 

06 

(08) 

06 

(08) 

05 

(07) 

06 

(05) 

11 

(09) 

08 

f07) 

05 

(07) 

10 

(10) 

05 

(07) 

08 

(OE 

10 

(11) 

07 

(09) 

10 

(in; 

1  1 

(12) 

06 

(09) 

Note:  Results  corrected  for  range  restriction  and  adjusted  for  shrinkage  (Rozeboctn  fonrula  8).  Nuntoers  In 

parentheses  are  standard  deviations.  Nuntocrs  In  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Decimals  omitted. 

•  PI  ratings  are  from  raters  in  Peer  Group  I;  P2  ratings  are  from  raters  in  Peer  Group  2:  and  SI  ratings  are 
from  raters  in  Supervisor  Croup  1. 

Number  of  MOS  for  which  validities  were  conputed. 
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Table  3.18 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  ASVAB  Factors,  Spatial, 
Computer,  JOB,  ABLE  Composites,  and  AVOICE  as  Predictors  of  Three  Combinations 


of  Single-Rater  Peer  and  Supervisor  End-of-7 raining  Ratings 


Criterion* 

MOS 

No.  of 
MOS** 

ASVAB 

Factors 

[4] 

Spatial 

(1] 

Con|>uter 

rs) 

JOB 

[3] 

ABLE 

Conp. 

[7] 

AVOICE 

[8] 

P1P2-ETS 

Batch  A 

10 

37  (07) 

30  (05) 

31  (07) 

20  (05) 

16  (10) 

21  (09) 

All  MOS 

21 

39  (12) 

34  (10) 

31  (11) 

19  (10) 

21  (14) 

21  (15) 

P1P2-NP0 

Batch  A 

10 

21  (04) 

18  (04) 

14  (07) 

07  (06) 

18  (06) 

09  (07) 

All  MOS 

21 

22  (13) 

19  (09) 

12  (09) 

Oh  (11) 

21  (11) 

08  (12) 

P1P2-PF8 

Batch  A 

10 

14  (09) 

06  (05) 

09  (09) 

04  (05) 

24  (03) 

07  (07) 

All  MOS 

21 

17  (14) 

09  (12) 

11  (12) 

05  (08) 

23  (11) 

09  (11) 

P1P2-LEAD 

Batch  A 

10 

26  (11) 

21  (08) 

23  (11) 

»5  (10) 

17  (11) 

16  (10) 

All  MOS 

21 

26  (16) 

24  (11) 

22  (16) 

16  (12) 

18  (14) 

16  (14) 

PlSl-ETS 

Batch  A 

10 

32  (06) 

27  (03) 

27  (07) 

IS  (09) 

1.’  (12) 

16  (12) 

All  MOS 

21 

36  (15) 

30  (13) 

28  (15) 

16  (11) 

16  (15; 

16  (13) 

PlSl-MPO 

Batch  A 

10 

19  (05) 

16  (05) 

12  (09) 

06  (07) 

11  (08) 

09  (06) 

All  MOS 

21 

22  (11) 

17  (09) 

n  (12) 

07  (12) 

15  (11) 

08  (08) 

PlSl-FPS 

Batch  A 

10 

13  (08) 

07  (041 

10  (06) 

05  (05) 

20  (09) 

08  (07) 

All  HOS 

21 

17  (17) 

13  (13) 

14  (16) 

06  (06) 

21  (12) 

07  (08) 

PlSl-LEAD 

Batch  A 

10 

23  (10) 

19  (07) 

19  (11) 

13  (09) 

17  (09) 

13  (11) 

All  MOS 

21 

27  (12) 

23  (11) 

18  (13) 

14  (10) 

17  (13) 

12  (11) 

PPS-ETS 

Batch  A 

10 

37  (05) 

31  (03) 

32  (06) 

20  (07) 

16  (11) 

19  (13) 

All  MOS 

21 

42  (14) 

35  (10) 

33  (11) 

20  (11) 

22  (13) 

21  (15) 

PPS-MPD 

Batch  A 

10 

22  (04) 

19  (04) 

16  (09) 

08  (06) 

15  (08) 

11  (08) 

All  MOS 

21 

24  (13) 

20  (08) 

14  (11) 

09  (12) 

18  (10) 

09  (10) 

PPS-PFB 

Batch  A 

10 

13  (09) 

07  (05) 

11  (07) 

05  (05) 

25  (07) 

09  (08) 

All  HOS 

21 

18  (16) 

11  (13) 

15  (13) 

05  (07) 

24  (10) 

09  (10) 

PPS-LEAO 

Batch  A 

10 

27  (09) 

22  (07) 

23  (10) 

15  (09) 

19  (10) 

16  (11) 

All  MOS 

2! 

29  (14) 

25  (12) 

22  (IS) 

18  (12) 

19  (13) 

14  (13) 

Note:  Results  corrected  for  range  restriction  and  adjusted  for  shrinkage  (Roreboom  fomila  8).  Numbers  in 
parentheses  are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of  predictor  scores 
entering  prediction  equations.  Decimals  omitted. 

*  P1P2  ratings  are  averages  of  ratings  from  raters  in  Peer  Groups  1  and  2:  PlSl  ratings  are  averages  of 
ratings  from  raters  in  Peer  Group  1  and  Supervisor  Group  1;  and  PPS  ratings  are  averages  of  ratings  from 
raters  in  Peer  Group  1.  Peer  Group  2.  and  Supervisor  Group  1. 

**  Nuirber  of  MOS  for  which  validities  were  computed. 


•  The  multiple  correlations  between  the  ASVAB  and  the  School 
Knowledge  Test  scores  were  quite  high.  The  "complete"  forms 
of  the  ASVAB  (i.e.,  the  nine  ASVAB  subtests  and  the  four  ASVAB 
factor  composite  scores)  performed  a  little  better  than  the 
"reduced"  forms  (i.e.,  the  AFQT  and  the  AA  composite  scores). 

•  The  ABLF  factor  scores  based  on  114  items  predicted  the  end- 
of-train^ng  performance  measures  slightly  better  than  both  the 
rationally  constructed  ABLE  composites  and  the  factor  scores 
based  on  168  items. 

•  For  the  most  part,  little  incremental  validity  was  found  for 
the  Experimental  Battery  over  the  ASVAB  with  respect  to  the 
School  Knowledge  tests.  Note,  however,  that  the  ASVAB 
baseline  multiple  correlations  for  the  School  Knowledge  tests 
are  very  high,  making  it  difficult  to  improve  upon  them. 

On  the  other  hand,  more  substantial  incremental  validity  was 
obtained  for  predicting  the  end-of-training  ratings.  Most  of 
this  increment  was  attributable  to  the  ABLE  when  regressed 
against  the  average  peer  ratings.  These  increments  ranged 
from  2  to  over  20  points,  depending  on  the  particular  rating 
composite  and  ASVAB  scores  used. 

In  addition,  more  incremental  validity  was  observed  when  the 
"reduced"  ASVAB  scores  served  as  the  baseline,  but  this  was 
due  primarily  to  the  lower  multiple  correlations  of  these 
ASVAB  scores  with  the  criterion  measures. 

•  As  expected,  comparison  of  results  corrected  and  uncorrected 
for  multivariate  range  restriction  suggests  that  the  same 
patterns  of  correlations  are  ootained  in  both  situations. 

With  regard  to  the  effect  of  the  correction  on  the  incremental 
validity  of  the  Experimental  Battery  over  the  various  ASVAB 
scores,  the  correction  yields  conservative  estimates  of  the 
incremental  validity  obtained  when  predicting  the  end-of- 
training  criterion  measures. 

•  Better  prediction  is  obtained  for'  the  peer  ratings  than  for 
the  supervisor  ratings,  even  when  one  controls  for  the  effects 
of  the  number  of  raters  per  ratee.  This  is  consistent  with 
the  prior  analysis  of  the  EOT  performance  measures  themselves, 
which  demonstrated  a  greater  degree  of  construct  validity  for 
the  peer  ratings  than  for  the  supervisor  ratings.  The  most 

.  reasonable  explanation  is  the  greater  opportunity  for  peers  to 
observe  the  relevant  components  of  performance. 

Future  validation  analyses  using  the  end-cf-training  criteria  will  focus 
on  the  degree  to  which  prediction  equations  generalize  across  MOS  for  specific 
criteria,  and  the  degree  to  which  the  equations  genera’ ’ze  across  criteria 
within  an  MOS.  That  is,  these  analyses  will  investigate  whether  these 
prediction  situations  require  MOS-specific  or  criterion-specific  rriression 
equations,  or  whether  a  single  regression  equation  is  sufficient  for  the 
purpose . 
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Chapter  4 

DEVELOPMENT  OF  BASIC  SCORES  FOR  THE  LVI  PERFORMANCE  MEASURES 

Walter  C.  Borman,  Charlotte  H.  Campbell,  Glenn  L.  Hallum, 
Mary  Ann  Hanson,  and  Deirdre  J.  Knapp 


In  1988  and  1989,  first-tour  criterion  measures  were  administered  to  the 
Longitudinal  Validation  sample  (LVI).  This  data  collection  was  conducted 
concurrently  with  the  administration  of  second-tour  criterion  measures  to  the 
Concurrent  Validation  sample  (CVII).  Before  we  could  begin  LVI  performance 
model  development  and  subsequent  validation  analyses,  it  was  necessary  to 
derive  basic  scores  for  each  of  the  individual  first-tour  job  performance 
measures.  Table  4.1  lists  the  individual  measures  that  were  administered. 

This  chapter  will  describe  the  procedures  that  were  used  to  develop  the 
basic  scores  for  each  measure,  present  the  results  of  the  score  development 
analysis,  and  portray  the  final  array  of  basic  scores  that  were  used  in  all 
subsequent  analyses  of  the  LVI  data.  Again,  the  major  reason  for  the  basic 
score  analysis  is  that  dealing  with  all  the  individual  scores  from  each  task 
test,  each  rating  scale,  and  each  administrative  index  is  simply  not  feasible 
or  desirable.  There  are  too  many  of  them,  there  is  much  redundancy,  and  the 
reliabilities  of  the  individual  items  or  scales  preserve  too  much  measurement 
error  with  very  little  gain  in  total  information. 

Table  4.1 


Performance  Measures  Administered  to  Soldiers  in  LVI  Sample _ 

MOS  in 

Batch  A:  Background  Information  Form 

Job  Knowledge  Tests 
Hands-on  Tests 
Army-Wide  Rating  Scales 
MOS-Specific  Rating  Scales 

Combat  Performance  Prediction  Scales  (males  only) 

Personnel  File  Form 

Army  Job  Satisfaction  Questionnaire 

Job  History  Questionnaire 

Physical  Requirements  Survey 

MOS  in 

Batch  Z:  Background  Information  Form 

School  Knowledge  Test 
Army-Wide  Rating  Scales 

Combat  Performance  Prediction  Scales  (males  only) 

Personnel  File  Form 

Army  Job  Satisfaction  Questionnaire 

Physical  Requirements  Survey 


Note.  Rating  scale  data  were  collected  from  both  supervisors  and  peers.  The  Physical  Requirements 
Survey  Is  not  a  Career  force  or  Project  A  measure. 
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Before  score  development  for  each  measure  is  discussed  in  turn,  the 
general  characteristics  of  the  measures,  the  LVI  data  collection  procedure, 
and  the  final  samples  that  were  obtained  will  be  described.  The  basic  data 
editing  and  imputation  procedures  used  to  generate  the  final  data  set  that  was 
used  in  the  performance  modeling  and  validation  analyses  will  be  described  in 
the  next  chapter. 

Because  the  LVI  measures  did  not  differ  greatly  from  the  CVI  instru¬ 
ments,  the  procedures  used  to  compute  basic  scores  for  the  LVI  criterion 
measures  were  quite  similar  to  those  used  to  score  the  CVI  criterion  measures. 
The  differences  that  do  exist  between  the  measures  administered  in  CVI  and 
those  administered  in  LVI  are  summarized  below. 


DIFFERENCES  BETWEEN  CVI  AND  LVI  PERFORMANCE  MEASURES^ 

The  three-year  time  period  between  CVI  and  LVI  raised  the  issue  that 
some  criterion  content  might  have  become  outdated.  Equipment  and/or 
procedural  changes  would  require  test  revisions,  and  changes  in  MOS  responsi¬ 
bilities  had  the  potential  of  making  some  tasks  obsolete. 

Project  staff  identified  relevant  changes  so  that  the  appropriate 
revisions  could  be  made.  In  a  few  cases  where  an  entire  task  was  obsolete, 
the  task  was  dropped  without  replacement.  In  many  cases,  revisions  were 
simply  a  matter  of  replacing  outdated  terminology.  Updated  criterion  measures 
were  forwarded  to  the  MOS  proponents  for  a  currency  review  and  additional 
revisions  were  made  on  the  basis  of  this  review. 

The  time  period  between  the  two  data  collections  was  particularly 
significant  for  the  19E  (M60  Armor  Crewman)  MOS  because  this  MOS  was  being 
severely  reduced  as  the  19K  (Ml  Armor  Crewman)  MOS  was  being  phased  in.  To 
deal  with  this  equipment/MOS  transition,  a  job  analysis  of  19K  was  conducted 
and  a  complete  set  of  criterion  measures  was  developed  for  this  MOS.  The  same 
procedures  used  for  the  other  MOS  (Campbell,  1987b)  were  followed,  with  one 
exception.  The  19K  MOS-specific  rating  scales  were  developed  by  SMEs  from  the 
Armor  School  and  by  19E  NCOs.  Because  of  the  19E/K  split,  the  LVI  data 
collection  included  10  MOS  in  Batch  A  rather  than  nine,  as  in  Project  A 
earl ier  work . 

Lessons  learned  from  CVI  prompted  the  use  of  a  different  format  for  the 
hands-on  test  sheets.  Additionally,  an  overall  effectiveness  rating  for 
performance  of  each  task  (on  a  scale  of  1  to  7)  was  added  to  the  end  of  each 
task  score  sheet  in  the  expectation  that  it  would  provide  unique  task 
performance  information. 

After  a  search  for  additional  first-tour  measures  that  would  have 
direct  relevance  to  the  issue  of  combat  readiness,  a  computer-simulated  M16 
rifle  marksmanship  task  was  selected  for  administration  to  MOS  IIB  and  95B 
soldiers  for  one  of  the  tasks  they  had  selected  for  testing.  Engage  Targets 
With  an  M16.  The  Multipurpose  Arcade  Combat  Simulator  (MACS)  was  originally 
developed  for  use  as  a  training  aid.  Using  a  demilitarized  M16  rifle,  the 


The  text  of  this  section  is  adapted  from  the  Project  A  Final  Report  (ARl  Research  Report  1597) 
edited  by  Campbell  and  Zook  (1991). 
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soldier  "shoots"  at  a  series  of  30  targets  displayed  on  a  computer  monitor, 
some  moving  and  some  stationary.  Attached  to  the  barrel  of  the  rifle  is  a 
light  pen  which  simulates  the  path  of  the  rounds. 

While  there  was  considerable  interest  in  keeping  the  Combat  Performance 
Prediction  Scales,  project  staff  and  the  Scientific  Advisory  Group  agreed  that 
the  version  used  in  CVI  was  too  lengthy.  Two  alternatives  were  considered. 

The  first  option  was  simply  to  reduce  the  number  of  items  in  the  original 
summated  rating  scale.  The  second  option  was  to  reduce  the  specific 
behavioral  items  to  summary  dimensions.  Three  dimensions  were  derived  through 
empirical  and  rational  analysis,  and  the  new  scales  were  field  tested  in 
conjunction  with  the  second-tour  criterion  measure  field  tests.  Low 
reliability  estimates  for  the  dimensional  ratings  led  to  the  decision  to 
retain  the  original  summated  scale  format,  but  the  total  number  of  items  was 
reduced  from  40  to  19. 

The  final  set  of  Combat  Performance  Prediction  Scale  items  was  selected 
by  considering  interrater  reliability,  internal  consistency,  and  content 
coverage.  That  is,  an  item  was  dropped  if  its  content  was  covered  in  another 
item  whose  estimated  reliability  was  higher,  or  if  its  content  was 
specifically  technical  in  nature  (and  thus  more  appropriately  covered  by  the 
other  criterion  measures).  Three  of  the  original  items  were  deleted  because 
SMEs  (senior  NCOs  with  combat  or  tactical  field  experience)  indicated  that 
they  were  not  meaningful.  Another  change  from  the  CVI  version  of  the  scales 
was  to  use  a  less  cumbersome  7-point  scale  instead  of  a  15-point  scale. 

Finally,  it  was  decided  that  Combat  Performance  Prediction  Scale  ratings  would 
not  be  collected  on  female  soldiers  since,  by  policy,  women  are  excluded  from 
participation  in  combat  situations.  (Note  that  this  decision  was  reversed  for 
the  LVII  data  collection,  following  the  central  involvement  of  female  soldiers 
in  Operations  Just  Cause  and  Desert  Storm.) 

The  self-report  form  for  gathering  information  on  administration  records 
was  updated  by  reviewing  its  contents  with  officers  and  NCOs  representing  the 
Ar.iiy  Personnel  Command  (PERSCOM).  The  form  was  altered  to  allow  soldiers  to 
report  an  M19  qualification  in  the  event  tnat  an  Hi6  qualification  is  not 
applicable.  Also,  three  awards  were  dropped  per  guidance  from  PERSCOM. 

Task-level  ratings  were  deleted  from  the  array  of  Batch  A  first-tour 
criterion  measures  used  in  CVI,  The  deletion  included  MOS-specific  tasks  that 
haa  appeared  in  the  MOS-specific  rating  scales  and  common  soldiering  tasks  that 
had  appeared  in  the  Army-wide  rating  scales.  The  task  ratings  were  eliminated 
because  they  were  not  sufficiently  reliable. 

Two  non-criterion  measures  that  had  been  administered  in  CVI  were  also 
dropped  from  the  LVI  administration.  The  deleted  instruments  were  the 
Measurement  Method  Rating,  which  allowed  soldiers  to  rate  the  fairness  of  each 
measurement  method  used,  and  the  Army  Work  Environment  Questionnaire. 

A  new  measure  developed  by  the  ARI  staff  was  the  Army  Job  Satisfaction 
Questionnaire  (AJSQ).  It  was  designed  to  provide  information  that  would  be 
potentially  useful  for  predicting  attrition  and  for  understanding  the 
relationship  of  job  satisfaction  with  other  constructs  investigated  within 
this  research  project.  Development  procedures  are  described  in  Campbell  and 
Zook  (1990).  the  AJSQ  was  administered  to  both  first-  and  second-tour 
soldiers.  The  derivation  of  a  scoring  scheme  and  associated  descriptive  data 
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analyses  for  these  LVI/CVII  data  are  provided  in  the  first  annual  report  for 
Career  Force,  and  will  not  be  presented  further  here. 


LVI  SAMPLE  DESCRIPTION 

Soldiers  from  22  MOS  were  included  in  the  administration  of  first-tour 
criterion  measures  in  1988  and  1989.  These  MOS  are  listed  in  Table  4.2.  The 
Batch  A  MOS  are  the  same  as  those  studied  in  the  Concurrent  Validation,  except 
for  the  addition  of  19K  (Ml  Armor  Crewman).  This  MOS  was  added  to  the 
research  in  response  to  the  planned  phasing  out  of  19E  (M60  Armor  Crewman). 

The  Batch  Z  MOS  differ  from  those  examined  in  the  Concurrent  Validation  in 
that  two  MOS,  29E  and  96B,  were  added  to  broaden  coverage  of  job  types  and 
one  MOS  (76W)  was  dropped  because  it  was  highly  redundant  with  another  (76Y). 


Table  4.2 


MOS  in  Longitudinal  Validation  Sample 


Batch  A 

Batch  Z 

lie 

Infantryman 

12B 

Combat  Engineer 

13B 

Cannon  Crewman 

15S 

MANPADS  Crewman 

19E 

M60  Armor  Crewman 

27H 

Tow/Dragon  Repairer 

19K 

Ml  Armor  Crewman 

29E 

Electronics  Repairer 

31C 

Single  Channel  Radio  Operator 

51B 

Carpentry/Masonry  Specialist 

63B 

Light-Wheel  Vehicle  Mechanic 

54B 

NBC  Specialist 

71L 

Administrative  Specialist 

55B 

Ammunition  Specialist 

88M 

Motor  Transport  Operator 

67N 

Utility  Helicoptor  Repairer 

91A 

Medical  Specialist 

76Y 

Unit  Supply  Special ist 

95B 

Mi  1  i tary  Police 

94B 

Food  Service  Specialist 

96B 

Intel  1  igence  Analyst 

As  in  CVI ,  the  Batch  A  MOS  differed  from  the  Batch  Z  MOS  in  the 
comprehensiveness  of  the  MOS-specific  criterion  measures  that  were  available 
for  administration.  MOS-specific  rating  scales,  hands-on  tests,  and  job 
knowledge  tests  were  administered  to  Batch  A  soldiers.  The  only  MOS-specific 
measure  available  for  administration  to  the  Batch  Z  soldiers  was  the  school 
knowledge  test  that  had  been  developed  for  administration  at  the  end  of 
training,  The  school  knowledge  test  was  administered  to  the  Batch  Z  examinees 
ds  a  surrogate  for  a  job  knowledge  test. 

Data  Collection  Strategy 

The  LVI  data  collection  strategy  has  been  described  elsewhere  (Campbell 
&  Zook,  1990),  but  will  pg  briefly  reviewed  here.  Project  staff  were  sent  to 
13  Army  installations  throughout  the  continental  United  States  and  to  several 
sites  throughout  Germany  to  collect  first-tour  job  performance  data.  The 
duration  of  each  test  site  visit  averaged  about  four  weeks,  and  the  data 
collection  spanned  July  1988  through  February  1989. 


Considerable  advance  coordination  was  necessary  to  ensure  that  each 
test  site  was  prepared  to  accommodate  the  data  collection.  In  addition  to 
provision  of  facilities  and  equipment,  examinees  and  their  supervisors  had  to 
be  identified  and  scheduled  for  participation.  As  it  turned  out,  the 
tracking,  scheduling,  and  testing  of  soldiers  who  had  been  tested  on  the 
Project  A  predictors  at  entry  proved  to  bf  extremely  challenging.  Indeed,  to 
ensure  that  sufficient  numbers  of  soldiers  were  tested  during  the  course  of 
the  data  collection,  it  was  decided  that  "suppleniental"  soldiers  would  have  to 
bo  tested  as  well.  Supplemental  examinees  are  defined  as  soldiers  who  entered 
the  Army  within  the  same  Basic  Active  Service  Date  window  as  "target"  soldiers 
(i.e.,  between  20  Aug  86  and  20  Nov  87),  but  from  whom  no  Project  A  predictor 
data  had  been  collected. 

Generally,  each  test  site  team  was  composed  of  a  Test  Site  I'lanager,  two 
Hands-On  Managers,  two  Hands-On  Assistants,  and  five  test  administrators. 
Additionally,  the  Army  installations  provideu  eight  NCOs  per  Batch  A  MOS  to 
administer  and  sccre  the  hands-on  tests.  Project  staff  comprising  the  data 
collection  teams  were  provided  with  both  formal  and  informal  training  related 
to  their  data  collection  responsibilities. 

The  written  test  component  (e.g.,  job  knowledge  or  school  knowledge  test, 
peer  ratings)  required  one-half  day  to  administer  and  the  hands-on  component 
required  another  one-half  day.  Thus,  Batch  A  soldiers  participated  for  one  full 
day  whereas  Batch  Z  soldiers  participated  tor  only  one-half  day  each. 

Sample  bizes 

Table  4.3  summarizes  sample  sizes  by  MOS  and  by  the  amount  of  pr  or  data 
that  are  available  for  the  LVI  examinees.  As  mentioned  above,  some  soldiers 
tested  in  LVI  had  not  previously  participated  in  Project  A  testing.  The  data 
provided  by  these  soldiers  are  useful  for  performance  modeling  purposes  and  to 
validate  ASVAB,  but  validation  analyses  related  to  the  Experimental  Battery  or 
the  end-of -training  measures  cannot  be  performed  with  their  data. 

Gut  of  the  11,266  soldiers  tested  in  LVI,  23  percent  liod  not  been  tested 
previously  ^js  part  of  this  research  program.  These  ore  the  so-called 
"supplemental"  examinees.  The  percentage  of  examinees  that  are  supplemental 
varies  considerably  acr-oss  MOS.  The  worst  case  is  with  MOS  19E,  in  which  63 
percent  of  the  examinees  had  not  been  tested  on  either  the  Experimental  Battery 
or  end-of “training  tests.  This  was  probably  due  to  factors  associated  with  the 
phasing  out  of  this  MOS.  The  best  case  is  with  MOS  IIB,  in  which  only  5  percent 
of  the  soldiers  tested  did  not  have  Experimental  Battery  and.'Or  training  data. 

It  is  also  interesting  to  note  that,  while  most  MOS  haa  a  trivial  number 
of  examinees  who  had  been  tested  previously  only  at  the  end  of  training  (in 
contrast  to  at  entry),  some  MOS  had  a  relatively  large  number  of  such 
examinees.  In  particular,  over  one-quarter  of  the  27E,  29E ,  and  31C  exaniinees 
fit  tnis  category. 

The  data  analyses  reported  in  this  chapter  for  score  development  are 
based  on  data  from  the  entire  LVI  sample,  regardless  of  the  amounts  of 
predictor  or  training  data  that  are  available  for  the  examinees.  Analyses 
reported  in  other  chapters  (e.g.,  Chapter  7),  however,  mav  be  based  on  subsets 
of  these  examinees  depending  on  the  type  of  variables  that  ere  required  for 
the  analysis.  Also,  the  data  files  used  to  develop  the  basic  scores  for  each 
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measure  were  subjected  only  to  the  preliminary  editing  described  by  each 
investigator.  The  score  development  analyses  were  conducted  only  on  cases 
with  complete  data.  The  data  files  that  were  later  used  for  the  modeling  and 
validation  analyses  were  subjected  to  the  final  editing  and  missing  data 
imputation  procedures  described  in  the  next  chapter.  This  chapter  sequence  is 
necessary  because  the  imputation  procedures  were  applied  at  the  basic  score 
level  for  some  instruments. 

Table  4.3 

LVI  Samole  Sizes  by  Amount  of  Prior  Data 


Predictor  Predictor  £r>d-of-  No  Prior 

End-of-Tralning  Only  Training  Only  Data 


111 

12.21 


13  :  232 

1.55  !  27.59 


30 

1 

773  1 

15  1 

13  : 

115  : 

916 

1 

1 

•  « 

84.39  1 

1.64  1 

1.42  1 

12.55  ! 

8.13 

57.20 


13.14 


117 

24.79 


60 

24.10 


29 

11.65 


157 

63.05 


568 

68.93 


225 

27.31 


37  : 
41.11  1 


31.11 


18 

20.00 


?9E 

I 

1 

1 

1 

44  ; 
39.29  1 

21 

18.75 

1 

1 

1 

f 

29 

25.89 

1 

1 

1 

1 

18 

16.07 

1 

1 

1 

1 

112 

.99 

31C 

1 

1 

1 

.  . 

154  1 
29.11  1 

94 

17.77 

1 

1 

1 

( 

144 

27.22 

1 

I 

1 

1 

137 

25.90 

1 

1 

1 

1 

529 

4.70 

5iB 

1 

1 

1 

I 

86  1 
40.38  1 

11 

5.16 

1 

1 

1 

1 

5 

2.35 

1 

1 

1 

1 

111 

52.11 

I 

1 

1 

1 

213 

1.89 

S-iB 

1 

\ 

I 

251  1 
50.30  1 

108 

21.64 

1 

1 

1 

19 

3.81 

1 

1 

1 

1 

121 

24.25 

f 

1 

> 

\ 

499 

4.43 

55B 

1 

1 

1 

138  1 
49.46  1 

11 

3.94 

1 

1 

1 

1 

58 

20.79 

1 

I 

I 

1 

72 

25.81 

t 

I 

\ 

I 

279 

2.48 

63B 

1 

1 

1 

307  1 
40.82  : 

263 

34.97 

1 

l 

1 

1 

20 

2.66 

1 

1 

1 

1 

162 

21.54 

1 

1 

1 

I 

752 

6.67 

(Continued) 
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Table  4.3  (Continued) 


LVI  Sample  Sizes  by  Amount  of  Prior  Data 


Predictor  Predictor  End-of-  Ho  Prior 

MOS  £nd-of-Training  Only  Training  Only  Data 


67N 


71L 


76Y 


88M 


9iA 


94B 


95B 


96B 


59 

29.95 


291 

42.92 


456 

57.87 


282 

41.35 


582 

70.63 


399 

47.96 


292 

64.60 


24 

12.18 


66 

9.73 


192 

24.37 


31 

4.55 


88 

10.68 


305 

36.66 


28 

6.19 


66 

51.56  ; 


31 

24.22 


34 

17.26 


80 

40.61 


46 

6.78 


275 

40.56 


3.43 


113 

14.34 


108  ! 
15.84  I 


261 

38.27 


71 

8.62 


83 

10.07 


22  ! 
2.64  : 


106 

12.74 


2 

0.44 


130 

28.76 


10 

7.81 


21 

16.41 


Total  N 
Percent 


6,401 

56.82 


1,532 

13.60 


730 

6.48 


2,603 

23.10 


Row  percentage 
Column  percentage 


Total 

197 

1.75 

678 

6.02 

788 

6.99 

682 

6.05 

824 

7.31 

832 

7.39 

4o2 

4.01 

128 

1.14 

11,266 

100.00 


SCORE  DEVELOPMENT  FOR  ADMINISTRATIVE  INDICES 

Five  scores  were  computed  from  the  CVI  Personnel  File  Form:  (a)  award 
and  memoranda/certificates  of  achievement,  (b)  Physical  Readiness  Test,  (c) 
M16  qualification,  (d)  Articles  15  and  flag  actions,  and  (e)  promotion  rate. 
Since  these  scores  captured  all  of  the  information  collected  on  the  form 
(except  for  the  Skill  Qualification  Tesc  score),  there  was  little  reason  to 
change  their  composition  for  L^I  purposes.  Some  changes,  however,  were  made 
to  the  way  in  which  variables  had  been  combined  to  form  the  composites  in  the 
CVI  analyses.  Each  of  the  five  first-tour  Personnel  File  Form  scores  is 
discussed  in  turn  below. 

The  first  score  from  the  Personnel  File  Form  is  a  composite  of  several 
items  thav  refect  recognition  of  exceptional  job  performance.  These  are 


(a)  awards  and  decorations;  (b)  memoranda  of  appreciation,  commendation,  or 
achievement;  and  (c)  certificates  of  appreciation,  commendation,  or 
achievement.  Soldiers  receive  promotion  board  credit  for  these  honors. 

Within  the  promotion  board  system,  the  Army  weights  awards  and  decorations 
depending  upon  the  relative  significance  of  the  various  honors.  Whereas  all 
awards  and  decorations  were  weighted  equally  in  the  CVI  scoring  system,  the 
LVI  scoring  system  weighted  these  honors  according  to  the  weighting  system 
incorporated  into  the  promotion  process.  Project  staff  believed  that  the  non¬ 
unit  weighting  scheme  would  more  appropriately  reflect  the  degree  of 
exceptional  performance  exhibited  by  the  soldiers  in  our  sample.  Note  that 
this  weighting  strategy  was  also  used  with  the  CVII  version  of  the  Personnel 
File  Form. 

In  another  variation  from  the  CVI  scoring  system,  the  composite 
"positive  recognition"  score  was  a  simple  sum  of  the  three  basic  scores  (i.e., 
awards,  memoranda,  and  rertif icates).  An  examination  of  the  distribution  of 
basic  scores  indicated  that  there  was  no  compelling  reason  to  standardize  the 
scores  before  combining  them  to  produce  the  composite.  Although  the  CVI 
scoring  system  standardized  the  basic  scci  s,  the  summing  of  raw  scores  was 
considered  to  be  more  parsimonious,  and  thus  preferaole  for  use  in  LVI. 

The  next  two  scores  from  the  Personnel  File  Fotm  are  the  result  of 
routine  tests  periodicilly  administered  by  the  Army.  The  first  is  the 
Physical  Readiness  Test  score.  Soldiers  generally  take  this  test  on  an  annual 
basis.  The  test  requires  the  soldier  to  perform  sit-ups,  push-ups,  and  a  2- 
mile  run.  The  M16  qualification  is  used  to  classify  soldiers  into  three  skill 
levels  of  rifle  firing  ability.  A  reasonably  large  number  of  soldiers  are 
assigned  the  M19  instead  of  the  M1.&,  which  is  why  the  M19  was  added  to  the  LVI 
Personnel  File  Form.  No  distinction  was  made  between  these  weapon  types  in 
scoring. 

Articles  15  and  flag  actions  reflect  disciplinary  problems.  In  CVI, 
the  frequency  of  each  type  of  action  was  standardized  prior  to  summing  to 
compute  the  disciplinary  action  composite  score.  As  with  the  "positive 
recognition"  composite  score  described  above,  it  was  determined  that  the 
distribution  of  scores  for  the  Articles  15  and  flag  actions  was  not  different 
enough  to  warrant  standardization  prior  to  summing. 

The  last  score,  promotion  rate,  does  not  actually  come  from  information 
collected  on  the  Personnel  File  Form,  Rather,  this  variable  is  derived  from 
data  available  in  the  Army's  computerized  personnel  records.  It  is  the 
.'esidual  of  pay  grade  regressed  cn  time  in  service,  adjusted  by  MOS. 

Descriptive  statistics  for  each  of  the  five  composite  variables  are 
shown  in  Table  4.4.  All  of  the  composite  scores  except  promotion  rate  are 
computed  in  the  same  manner  as  the  corresponding  scores  on  the  CVII  version  of 
the  Personnel  File  Form. 

As  one  would  expect,  second-tour  soldiers  had  more  awards/memoranda/ 
certificates  than  first-tour  soldiers  (an  average  of  10.5  compared  to  3). 

Their  average  PHvsiral  Readiness  Test  score  was  also  somewhat  higher  (250 
compared  to  238).  The  M16/M19  Qualification  score  was  only  slightly  higher 
for  second-tour  soldiers  (2.5  versus  2.2).  Somewhat  surprisingly,  second-tour 
soldieis  appeared  to  get  as  many  disciplinary  actions  in  the  second  tour  as 
*’irst-toiir  soldiers  we'"e  getting  ’n  the  first  tour  (.42  compared  to  .40). 
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Table  4.4 

LVI  Personnel  File  Fonii  Descriptive  Statistics 


Composite  Variable 

Mean 

SO 

N 

Minimum 

Maximum 

Awards/Memos/Certificates 

3.29 

3.18 

10,894 

0 

21 

Physical  Readiness 

238.30 

32.75 

10,389 

19 

300 

M16/M19  Qualification 

2.21 

.76 

10,735 

1 

3 

/'rticles  15/Flan  Act’^'es 

A  A 
•  T  Vi 

.88 

ii.uyb 

0 

8 

Promotion  Rate 

1.97 

.59 

10,915 

-2.85 

2.16 

SCORE  DEVELOPMENT  FOR  COMBAT  PERFORMANCE  PREDICTION  RATING  SCALES 

Ratings  on  the  Combat  Performance  Prediction  Scales  were  analyzed  to 
determine  the  preferred  method  for  scoring  this  instrument.  Given  that  a 
scoring  system  for  the  second-tour  version  of  these  scales  had  already  been 
devised,  and  that  the  same  items  would  be  scored  on  both  the  first-tour  and 
second-tour  versions  of  the  scales,  it  was  decided  that  the  second-tour 
scoring  system  would  be  used  unless  characteristics  of  the  first-tour  data 
suggested  that  this  would  be  inappropriate.  This  scoring  system  combined 
supervisor  and  peer  ratings,  and  produced  a  single  score  by  totaling  averaged 
ratings  across  the  14  scored  items.  This  differed  from  the  CVI  scoring 
system,  which  combined  supervisor  and  peer  ratings,  and  summed  items  to 
produce  two  scores,  "Performing  Under  Adverse  Conditions"  and  "Avoiding 
Mistakes. " 

Supervisor  ratings  were  collected  for  8,713  first-tour  soldiers  and  peer 
ratings  were  collected  for  8,153.  (Recall  that  all  soldiers  rated  on  these 
scales  were  male.)  A  total  of  91  percent  of  the  soldiers  had  ratings  from  at 
least  one  supervisor  and  68  percent  had  ratings  from  two  or  more  supervisors. 
Peer  ratings  were  not  as  complete,  with  85  percent  of  the  soldiers  having 
ratings  from  at  least  one  peer  and  55  percent  having  ratings  from  two  or  more 
peers . 


Principal  components  analyses  of  the  second-tour  Combat  Scale  data 
indicated  the  presence  of  two  factors.  The  second  factor,  however,  was 
defined  by  the  three  negatively  worded  items  on  the  rating  scale.  Results 
were  virtually  identical  with  the  analysis  of  the  first-tour  data.  Given 
that,  in  both  cases,  the  second  factor  is  probably  not  substantively  distinct 
from  the  first,  the  calculation  of  a  single  total  score  (with  the  negatively 
worded  items  reverse-scored)  for  the  Combat  Scale  ratings  appeared  appropriate 
for  both  first-  and  second-tour  ratings.  Note  that  the  two  factors  found  in 
the  LVI/CVII  data  are  essentially  the  same  as  those  which  were  found  in  CVI, 
and  used  to  derive  the  two  Combat  Scale  scores  at  that  time. 


Interrater  reliability  estimates  are  reported  in  Table  4.5.  These 
estimates  are  liigher  than  those  obtained  with  the  second-tour  rat^'^gs  because 


there  tended  to  be  more  ratings  for  each  first-tour  examinee.  Coefficient 
alphas  were  virtually  identical  for  ratings  of  first-  and  second-tour 
soldiers.  Supervisor  ratings  yielded  a  coefficient  of  .930,  peer  ratings 
yielded  a  coefficient  of  .906,  and  the  combined  ratings  exhibited  an  alpha 
coefficient  of  .934, 

Given  these  results,  the  decision  was  made  to  score  the  Combat 
Performance  Prediction  Scales  in  the  same  manner  for  both  CVII  and  LVI.  That 
is,  only  one  score  was  computed,  and  it  was  calculated  by  summing  the  combined 
supervisor  and  peer  ratings  across  all  items.  It  is  worth  noting  that  the 
mean  Combat  Scale  score  was  appreciably  higher  for  second-tour  soldiers  (mean 
=  70;  SO  =  11)  than  for  first-tour  soldiers  (mean  =  63;  SD  =  11). 

Table  4.5 

LVI  Combat  Scales  Interrater  Reliability  Estimates _ 

1-Rater  Rel iabi 1 ity  N-Rater  Rel iabi 1 itv 


Supervisor  Ratings 

.399 

.543 

Peer  Ratings 

.247 

.478 

Combined  Ratings 

.279 

.607 

DEVELOPMENT  OF  BASIC  SCORES  FOR  THE  FIRST-TOUR  PERFORMANCE  RATING  SCALES 

This  section  reports  results  of  the  analyses  of  first-tour  performance 
rating  data  for  the  Longitudinal  Validation  (LVI)  sample.  There  were  two 
major  objectives:  (a)  to  evaluate  the  psychometric  properties  of  the  ratings 
to  ensure  that  they  were  of  high  quality,  and  (b)  to  identify  a  set  of  rating 
scale  composites  (basic  scores)  that  would  appropriately  reflect  the  major 
dimensions  of  content  in  the  performance  ratings. 

As  reported  previously  (Campbell,  1987;  Pulakos  &  Borman,  1986;  Toquam 
et  al.,  1986),  Army-wide  and  MOS-specific  first-tour  performance  rating  scales 
were  developed  for  use  by  supervisors  and  peers  of  first-tour  soldiers  to 
evaluate  these  soldiers  on  the  important  dimensions  of  performance.  In 
particular,  the  behavioral ly  anchored  rating  scale  (BARS)  method  (Campbell, 
Dunnette,  Arvey,  &  Hellervik,  1973)  was  used  to  identify  soldier  effectiveness 
dimensions  and  to  develop  behavioral  definitions  of  performance  for  each 
dimension.  The  Army-wide  rating  scales  include  12  dimensions  of  soldier 
effectiveness  that  are  important  regardless  of  sol'-'ers’  MOS.  MOS-specific 
rating  scales  were  developed  for  each  of  the  nine  Batch  A  MOS,  and  these 
rating  scales  include  between  7  and  13  dimensions  of  MOS-specific  performance. 

The  Army-wide  and  MOS-specific  rating  scales  were  used  previously,  as 
part  of  the  Concurrent  Validation  study,  to  obtain  job  performance  information 
for  a  sample  of  8,642  first-tour  soldiers  (referred  to  as  the  CVI  sample)  from 
their  peers  and  supervisors.  Analyses  of  these  data  (Pulakos  &  Borman,  1987) 
showed  that  the  ratings  obtained  had  appropriate  distributions  and 
Sdtistactory  reliability,  and  factor  analyses  revealed  three  interpretable 
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underlying  dimensions  of  job  performance.  Discussion  of  the  present  data 
analyses  will  compare  these  CVI  results  with  those  obtained  for  the  LVI 
sample. 


Sample  Description 

We  attempted  to  obtain  multiple  peer  and  supervisor  ratings  for  each 
individual  in  the  LVI  target  sample,  which  consisted  of  6,602  first-tour 
soldiers  from  Batch  A  MOS  and  4,311  first-tour  soldiers  from  Batch  Z  MOS. 

Data  collection  administrators  and  the  Army  Points-of-Contact  identified  and 
contacted  supervisors  and  peers  of  the  target  sample  members.  Peer  ratings 
were  generally  obtained  only  when  peers  of  the  ratees  were  also  members  of  the 
target  sample.  In  a  few  cases,  when  no  supervisor  ratings  could  be  obtained, 
peers  who  were  not  members  of  the  target  sample  were  brought  in  to  maKe 
ratings.  Supervisor  and  peer  rating  sessions  were  usually  conducted 
separately. 

Tables  4.6  and  4.7  show,  by  MOS,  the  actual  numbers  of  supervisor  and 
peer  raters  who  provided  ratings  for  each  member  of  the  target  sample  for 
Batch  A  and  2  respectively.  As  shown  by  Table  4.6,  across  all  Batch  A  MOS, 
more  than  80  percent  of  the  target  sample  had  at  least  one  peer  rating  (5,450 
of  6,602)  and  more  than  90  percent  of  the  target  sample  had  at  least  one 
supervisor  rating  (6,022  of  6,602).  Those  Batch  A  ratees  who  had  at  least  one 
peer  rating,  there  were  an  average  of  2.82  peers  per  ratee.  For  those  who 
received  supervisory  ratings,  an  average  of  1.83  supervisors  per  ratee 
provided  ratings.  Table  4.7  shows  that,  across  all  Batch  Z  MOS,  more  than  85 
percent  of  the  target  sample  obtained  at  least  me  peer  rating  (3,672  of 
4,311);  more  than  90  percent  of  the  target  sample  obtained  at  least  one 
supervisor  rating  (3,903  of  4,311).  Those  Batch  Z  ratees  who  had  any  peer 
ratings  at  all  had  an  average  of  2.93  peers  per  ratee.  For  those  who  received 
supervisory  ratings,  there  were  an  average  of  1.82  supervisors  per  ratee. 

An  extremely  important  aspect  of  each  rating  session  was  a  rater 
orientation  and  training  program  devtloped  to  reduce  various  rating  errors 
(e.g.,  halo)  and  to  persuade  raters  to  provide  evaluations  that  are  as 
accurate  as  possible.  The  orientation/training  program  was  an  adaptation  of 
the  program  aevelop'=.c  for  raters  participating  in  the  Concurrent  Validation 
first-tour  data  coiiecticn  (Pulakos  &  Borman,  1986). 

Data  Analyses 

Before  the  basic  data  analyses  were  conducted  on  the  Army-wide  and  MOS- 
specific  ratings,  preliminary  analyses  assessed  the  quality  of  the  obtained 
ratings  and  explored  the  possibility  of  screening  out  poor-quality  data.  For 
each  rater  who  had  rated  at  lea'-.t  two  soldiers  comnon  to  other  raters,  a 
measure  of  the  proximity  of  tfieir  ratings  to  those  made  by  other  raters  was 
computed.  (Ratees  were  included  in  this  analysis  only  if  they  had  been  rated 
by  at  least  two  persons.)  It  was  expected  that  raters  who  had  made  their 
ratings  carelessly  or  incorrectly  would  provide  ratings  that  were,  on  average, 
quite  dissimilar  to  those  provided  by  others  who  had  rated  the  same  soldiers. 
This  proximity  measure  represents  the  average  discrepancy  between  ratings  made 
by  a  particular  rater  and  those  made  by  all  other  persons  who  had  rated  the 
same  soldier's.  Calculating  the  proximity  measure  involved  computing,  for  each 
rater  and  for  each  dimension,  the  absolute  difference  between  the  dimension 
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Table  4.6 

LVI  Army-Wide  Ratings;  Number  of  Peer  and  Supervisor  Ratings  per  Ratee  by 
HOS,  Batch  A _ 


MOS 


Total 


_ IIB  13B  19E  19K  31C  63B  71L  88M  91A  95B  Sample 

PEERS 

Number  of  Ratings 


0 

69 

74 

6 

141 

163 

192 

300 

86 

102 

19 

1152 

1 

110 

120 

37 

95 

114 

213 

168 

87 

115 

37 

1096 

o 

4 

242 

178 

79 

111 

103 

164 

108 

132 

141 

58 

1316 

3 

210 

241 

77 

141 

47 

99 

31 

122 

164 

110 

1242 

4 

190 

191 

34 

171 

57 

50 

25 

176 

187 

182 

1263 

5 

58 

67 

5 

88 

12 

9 

2 

46 

70 

38 

395 

6 

18 

17 

3 

27 

1 

1 

0 

16 

17 

6 

106 

7 

2 

6 

0 

4 

0 

0 

0 

0 

9 

1 

22 

8 

0 

3 

0 

2 

0 

0 

0 

1 

2 

0 

8 

9 

0 

0 

0 

2 

0 

0 

0 

0 

0 

0 

2 

LVI  Sample 
Total  N  899 

897 

241 

782 

497 

728 

634 

666 

807 

451 

6602 

Mean  Number  of 

Ratings 

.  per 

Ratee 

Total  LVI 
Sample  2.67 

2.75 

2.51 

2.67 

1.52 

1,50 

.93 

2.64 

2.69 

3.20 

2.33 

Rated 

Persons  2.89 

3.00 

2.57 

3.26 

2.26 

2.03 

1.76 

3.03 

3.08 

3.34 

2.82 

SUPERVISORS 

Number  of  Ratings 

0  66 

98 

15 

146 

61 

45 

38 

35 

58 

18 

580 

1  141 

176 

47 

164 

137 

236 

218 

127 

237 

89 

1572 

2  608 

564 

154 

350 

277 

423 

356 

473 

492 

323 

4020 

3  76 

45 

21 

44 

21 

22 

19 

30 

17 

20 

315 

4  6 

9 

3 

76 

1 

2 

3 

1 

3 

1 

105 

5  1 

5 

1 

2 

0 

0 

0 

0 

0 

0 

9 

6  1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

Mean  Number  of 

Ratings 

per 

Ratee 

Total  LVI 
Sample  1.80 

1.67 

1.80 

1.68 

1.53 

1.59 

1.58 

1.75 

1.59 

1.77 

1.67 

Rated 

Persons  1 .94 

1.88 

1.92 

2.06 

2.26 

1.69 

1.68 

1.85 

1.71 

1.85 

1.83 
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Table  4.7 


LVI  Army-Wide  Ratings:  Number  of  Peer  and  Supervisor  Ratings  per  Ratee  by 
HOS,  Batch  Z _ _ _ 


MOS 

-  Total 

12B  16S  27E  29E  51B  54B  55B  67N  76Y  94B  96B  Sample 


PEERS 

Number  of  Ratings 


0 

42 

9 

15 

26 

19 

52 

23 

15 

311 

81 

46 

639 

1 

106 

38 

17 

43 

32 

46 

31 

26 

205 

157 

37 

738 

2 

177 

58 

19 

14 

45 

56 

46 

43 

121 

189 

22 

790 

3 

218 

96 

14 

18 

52 

116 

71 

38 

63 

177 

11 

874 

4 

195 

154 

14 

4 

31 

118 

55 

49 

26 

125 

3 

774 

5 

64 

80 

5 

1 

11 

67 

29 

16 

6 

64 

3 

346 

6 

16 

26 

2 

3 

19 

10 

5 

2 

15 

0 

0 

98 

7 

8 

6 

0 

0 

0 

12 

4 

1 

0 

5 

0 

36 

8 

0 

1 

1 

0 

0 

4 

0 

0 

0 

3 

0 

9 

9 

1 

0 

1 

0 

0 

2 

0 

0 

0 

1 

0 

5 

10 

C 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

1 

11 

0 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

LVI  Sample 


Total  N 

827 

468  89 

106  193 

492 

269 

193 

734 

818 

122 

4311 

Mean  Number 

of  Ratings  per 

Ratee 

Total  LVI 

Sample 

2.87 

3.56  2.45 

1.38  2.46 

3.17 

2.93 

2.79 

1.07 

2.51 

1.16 

2,49 

Rated 

Persons 

3.03 

3.63  2.95 

1.83  2.73 

3.55 

3.21 

3.03 

1.85 

2.79 

1.86 

2.93 

SUPERVISORS 

Number  of  Ratings 

0 

71 

45  6 

6  19 

30 

25 

20 

72 

97 

17 

408 

1 

204 

109  5 

31  32 

122 

50 

57 

247 

212 

26 

1095 

2 

517 

279  76 

62  45 

312 

184 

110 

396 

456 

72 

2509 

3 

33 

30  2 

4  52 

23 

8 

4 

17 

43 

7 

223 

4 

2 

4  0 

3  31 

5 

2 

1 

1 

7 

0 

56 

5 

0 

0  0 

0  11 

0 

0 

1 

1 

3 

0 

16 

6 

0 

1  0 

0  3 

0 

0 

0 

0 

0 

0 

4 

Mean  Number 

of  Ratings  per 

Ratee 

Total  LVI 

Samp  1 e 

1.62 

1.66  1.83 

1.69  1.71 

1.70 

1.67 

1.54 

1.50 

1.58 

;  57 

1  . 65 

Rated 

Persons 

1.78 

1.84  1.96 

1.79  1.76 

!.81 

1.84 

1.72 

1.56 

1.30 

1.32 

•,  Q  9 

rating  made  by  that  particular  rater  and  the  mean  dimension  rating  made  by  all 
others  who  had  rated  that  particular  ratee.  The  proximity  measure  for  a 
particular  rater  was  then  simply  the  mean  of  these  absolute  differences  across 
all  dimensions  and  all  ratees. 

Next,  for  both  Army-wide  and  MOS-specific  performance  rating  scales, 
analyses  were  conducted  focusing  on  the  distributions  of  the  ratings  (e.g., 
means  and  standard  deviations)  and  interrater  reliabilities.  Analyses  for  the 
Army-wide  ratings  were  carried  out  on  the  total  sample  (including  all  Batch  A 
and  Batch  Z  soldiers);  MOS-specific  ratings  were  of  course  analyzed  separately 
by  MOS  (for  Batch  A  soldiers  only). 

Principal  factor  analyses  with  varimax  rotation  were  conducted  on  the 
Army-wide  ratings  (across  all  MOS),  for  supervisor  and  peer  ratings  separately 
and  pooled  together.  The  pooled  ratings  were  computed  by  averaging  the  mean 
peer  rating  and  the  mean  supervisor  rating  for  those  soldiers  who  had  at  least 
one  peer  rating  and  one  supervisor  rating.  Because  previous  analyses  (using 
the  CVI  sample)  showed  that  a  single  factor  was  sufficient  to  account  for  the 
majority  of  the  variance  in  the  MOS-specific  ratings,  factor  analyses  were  not 
conducted  for  the  MOS-specific  rating  data. 

Re.su  Its 


Data  Quality  Screening 

Over  68  percent  of  the  raters  for  whom  the  proximity  measure  screen  was 
computed  obtained  scores  of  1.2  or  less  on  this  screen.  This  indicates  that 
the  mean  absolute  difference  between  the  ratings  made  by  these  raters  and 
those  made  by  other  raters  was  1.2  or  less  on  a  7-point  scale.  For  more  than 
98  percent  of  the  raters  the  mean  absolute  difference  score  was  2.0  or  less. 

It  was  decided  not  to  screen  out  data  based  on  raters'  scores  on  this 
proximity  measure  for  three  reasons.  First,  the  fact  that  most  raters 
obtained  relatively  low  scores  on  this  measure  shows  that  in  general  the  data 
are  of  very  high  quality.  Second,  if  a  relatively  high  cut  score  was  selected 
(e.g.,  screen  out  those  scoring  2.0  or  higher  on  the  proximity  measurel,  very 
little  data  would  be  screened  out,  and  the  screening  would  have  little  impact 
on  the  results  of  the  remaining  data  analyses.  Third,  it  was  not  clear  what  a 
reasonable  cut  score  would  be  on  this  type  of  screening  measure. 

Army-Wide  Rating  Scales 

For  the  Army-wide  ratings,  data  are  first  presented  on  the  rating 
distributions.  Second,  interrater  reliability  results  are  shown;  and  finally, 
factor  analysis  results  are  summarized. 

Rating  distributions.  Table  4.8  displays  the  Army-wide  rating 
distributions,  showing  the  scale  points  utilized  by  supervisor  and  peer 
raters.  These  results  demonstrate  that  raters  are  employing  all  scale  points, 
although  the  "1"  and  "2"  points  are  used  minimally,  especially  by  peers. 


Table  4.8 


LVI  Army-Wide  Ratings:  Use  of  Scale  Points  by  Peer  and  Supervisor  Raters 
(percent) _ 


7 

6 

5 

4 

3 

2 

1 

Peer  Raters 


Technical  Knowledge/Sk i 11 

9.2 

25.7 

27.5 

21.3 

10.1 

4.4 

1.8 

Effort 

7.1 

16.4 

27.4 

22.4 

14.1 

8.9 

3.6 

Following  Regulations 

8.2 

19.0 

26.7 

21.6 

13.1 

7.8 

3.5 

Integri ty 

10.8 

20.4 

26.6 

19.9 

11.2 

7.3 

3.8 

Leadership 

5.9 

13.8 

22.2 

22.0 

17.9 

12.1 

6.1 

Maintaining  Equipment 

12.0 

22.6 

27.0 

20.2 

11.3 

4.7 

2.2 

Military  Appearance 

13.3 

21.8 

26.5 

18.6 

10.8 

5.8 

3.3 

Physical  Fitness 

12.2 

22.0 

28.1 

19.0 

10.7 

5.6 

2.6 

Self-Development 

6.1 

15.9 

26.4 

23.0 

15.8 

9.2 

3.6 

Self-Control 

12.2 

20.0 

24.1 

18.5 

12.5 

8.0 

4.7 

Overall  Effectiveness 

5.5 

18.9 

34.7 

23.2 

11.4 

4.5 

1.7 

NCO  Potentia  i 

8.8 

18.5 

23.3 

18.6 

13.5 

10.3 

7.0 

Supervisor  Raters 

Technical  Knowledge/Sk i 1 1 

6.2 

17.6 

29.4 

25.5 

14.3 

5.4 

1.1 

Effort 

7.5 

15.3 

25.0 

19.8 

15.4 

13.2 

3.7 

Fol  lowing  Regulations 

9.2 

18.7 

26.2 

19.7 

13.7 

9.3 

3.3 

I ntegr i ty 

11.5 

20.2 

27.0 

17.7 

11.6 

8.8 

3.2 

Leadership 

3.8 

9.6 

19.1 

20.8 

21.6 

17.8 

7.2 

Maintaining  Equipment 

10.0 

18.9 

25.5 

20.7 

15.5 

7.0 

2.4 

Military  Appearanre 

12.4 

19.0 

26.0 

18.4 

13.1 

8.3 

2.7 

Physical  Fitness 

Se  1  f-Development 

13.6 

20.1 

29.6 

17.5 

10.5 

6.2 

2.4 

4.5 

11.5 

23.6 

23.2 

19.7 

13.5 

3.9 

Self-Concrol 

13.5 

22.1 

23.7 

17.6 

11.7 

7.7 

3.6 

Overall  Effectiveness 

3.9 

17.0 

32.7 

24.0 

14.4 

6.6 

1.4 

NCO  Potent i a  1 

6.3 

18.6 

22.4 

18.6 

14.8 

12.5 

6.8 

Note.  Number  of  peer  raters 

ranges 

from  24 

,764 

to  25,882; 

;  number 

of 

supervisor  raters  ranges  from 

17,354 

to  1 

7,903. 

The  mesn  ratings  fr,-  this  LVI  sample  are  very  similar  to  those  obtained 
for  the  CVI  sample,  as  snown  in  Table  4.9.  In  addition,  the  peer  ratings  are 
slightly  higher  than  the  supervisor  ratings.  The  spread  or  differentiation  of 
the  ratings  across  ratees  is  indicated  by  the  standard  deviations,  which  are 
also  reported  on  Table  4.9.  On  average,  the  spread  of  the  ratings  obtained 
for  the  ..VI  sample  is  similar  to  that  obtained  for  the  CVI  sample.  The 
standard  deviation  of  the  supervisory  ratings  for  the  LVI  sample  is  quite 
comparable  with  that  of  the  peer  ratings  for  this  same  sample. 
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Overall,  the  distributions  of  supervisor  and  peer  ratings  for  the  LVI 
sample  seem  appropriate.  They  show  few  signs  of  errors  of  central  tendency  or 


leniency. 

.fible  4.9 

LVI  Army-Wide  Ratings:  Means  and  Standard  Deviations 


Peer 

Raters* 

Supervisor 

Raters^ 

Dimension 

Mean 

SO 

Mean 

SO 

Technical  Knowledge/Skill 

4.82 

1.36 

4.54 

1.31 

Effort 

4.39 

1.50 

4.25 

1.58 

Following  Regulations 

4.50 

1.51 

4.49 

1.55 

Integrity 

4.62 

1.55 

4.63 

1.57 

Leadership 

4.07 

1.58 

3.71 

1.55 

Maintaining  Equipment 

4.81 

1.45 

4.57 

1.50 

Mi  1 itary  Appearance 

4.78 

1.54 

4.63 

1.56 

Physical  Fitness 

4.79 

1.48 

4.81 

1.50 

Self-Development 

4.31 

1.48 

4.02 

1.48 

Self-Control 

4.58 

1.63 

4.71 

1.61 

Overall  Effectiveness 

4.64 

1.28 

4.47 

1.29 

NCO  Potential 

4.32 

1.68 

4.18 

1.66 

Mean  Across  All  Rating 
Dimensions 

4.55 

1.50 

4.42 

1.51 

^  N  =  24,764  -  26,095. 
N  =  17,354  -  17,993. 


Interrater  rel iabi 1 ity.  Interrater  reliability  results  for  the  Army¬ 
wide  scales  are  presented  in  Table  4.10.  These  are  shown  for  peer, 
supervisor,  and  pooled  peer  and  supervisor  ratings.  The  intraclass 
correlations  that  reflect  the  reliability  of  a  single  rater  are  labeled  r^jjj, 
and  the  intraclass  correlations  that  reflect  the  reliability  of  the  mean 
ratings  across  raters  are  labeled  Because  the  latter  intraclass 

correlations  depend  in  part  on  the  average  number  of  raters  per  ratee,  the 
one-rater  indices  are  best  for  comparing  the  relative  levels  of  interrater 
agreement  for  different  rater  groups. 
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Table  4,10 


LVI  Army-Wide  Ratings :  One 
Ratings 

-Rater  and 

k-Rater^  Reliabilities  of 

Dimension 

Peer 

Ratings'’ 

Supervisor 

Ratings*^ 

Pooled 

Peer/Sup. 

Rat  i  ngs‘’ 

Dimension 

■^(11) 

'^(11) 

’"(kk) 

Coeff ic ient 
Alpha 

Technical  Knowledge/Skill 

.25 

.49 

.39 

.53 

.56 

Effort 

.24 

.47 

.37 

.51 

.  52 

Following  Regulations 

.25 

.49 

.38 

.52 

.  56 

Integrity 

.19 

.40 

.34 

.49 

.48 

Leadersh ip 

.25 

.49 

.38 

.52 

.54 

Maintaining  Equipment 

.17 

.36 

.31 

.45 

.44 

Mi  1 i tary  Apoearance 

.29 

.54 

.44 

.59 

.62 

Physical  Fitness 

.33 

.59 

.48 

.63 

.65 

Self-Development 

.21 

.44 

.33 

.47 

.49 

Self  Control 

.23 

.46 

.34 

.48 

.53 

Overall  Effectiveness 

.25 

.48 

.41 

.  56 

.  56 

NCG  Potential 

.29 

.53 

.44 

.58 

.50 

Median  Across  Al 1 

Rating  Dimensions 

.25 

.48 

.38 

.52 

.55 

Median  Across  All  Rating 
Dimensions  for  CVI  Sample 

.22 

.49 

.37 

.52 

-- 

Note.  r,,jj  =  intraclass  correlations 
rater. 

'"(kkj  ~  intraclass  correlations 
ratings  across  raters. 

reflecting 

reflecting 

rel  iabi 1 ity  of 

reliability  of 

a  single 

the  mp^n 

^  !k  i  s  the  mean  number  of  ratings  per  ratee. 

^  The  average  number  of  peer  ratings  per  ratee  ranges  from  2.76  to  2.84. 

^  The  average  number  of  supervisor  ratings  per  ratee  ranges  from  1.77  to  1.,. 
lomputed  by  averaging  the  mean  peer  rating  and  the  mean  supervisor  rating; 
the  average  number  of  peer  or  supervisor  ratings  per  ratee  ranges  f^'orn  3.96 
to  4.1. 
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The  one-rater  reliafcnuies  on  Table  4.10  show  that  the  degree  of 
interrater  agreement  for  peers  and  supervisors  is  oltric-st  exactly  the  same  as 
was  found  in  th'^  CVI  research.  Also,  supervisors  again  provided  considerably 
more  reliable  tings  than  did  peers. 

Table  4.10  also  indicates  f  at  the  wodn  ratings  are  quite  reliable.  The 
reliability  of  the  mean  peer  rating  is  comparable  tc  that  of  the  rican 
supervisor  rating,  but  keep  in  mind  that  the  average  numbers  of  peer  ratings 
per  ratee  are  a  bit  higher  than  fo'  supervisors.  When  peer  and  supervisor 
ratings  are  pooled,  the  additional  numbers  of  raters  per  ratee  inc.-ease  ti.e 
level  of  reliability  for  the  combined  peer/supervi ior  ratings  to  a  level 
somewhat  higher  than  that  of  the  mean  supervisor  ratings. 

In  sum,  the  interrato'"  reliabilities  in  the  present  sample  are  rifidr'iy 
identical  tc  those  obtained  for  the  CVI  sample.  All  of  the  mean  ratings 
peer,  supervisor,  and  pooled  peer/supervisor  --  have  acceptable  levels  of 
rel iabi 1 itv. 

Factor  analysis  results.  Several  factor  analyses  were  conducted  on  the 
Army-wide  soldier  ratings  for  the  LVI  sample.  Ratings  made  by  peers  on  the  10 
behavior-based  rating  dimensions  were  intercorrelated  and  factor  analyzed,  and 
the  results  were  compared  with  the  facto-'  structure  for  the  ratings  of  the  CVI 
sample.  The  same  procedure  was  followed  for  the  supervisor  ratings  and  for 
the  pooled  peer  and  supervisor  ratings. 

Tables  4.11  -  4.13  show  the  three-factor,  rotated  solutions  for  peer, 
supervisor,  and  pooled  peer/supervisor  ratings  respectively.  These  tables 
demonstrate  the  remarkable  similarity  of  the  rotated  factor  structures  for  the 
CVI  and  LVI  samples.  With  one  small  exception,  the  three  factors  obtained  in 
the  CVI  sample  were  very  closely  replicated  with  the  LVI  data.  The  one 
exception  is  in  the  factor  analysis  of  the  peer  ratings  for  the  LVI  sample 
where  the  factors  emerged  in  a  different  order,  and  two  of  the  scales  --  Self- 
Development  and  Maintaining  Equipment  --  had  their  highest  load  ng  on  a 
different  factor  than  they  did  in  the  CVI  factor  analysis.  Even  so,  tiie 
values  of  the  factor  loadings  for  the  LVI  peer  ra^’ing  data  were  generally  very 
similar  to  those  obtained  for  the  CVI  data. 

It  is  worth  noting  that  these  same  three  factors  were  also  obtained  in 
factor  analyses  of  performance  rating  data  for  a  sample  of  950  second-tour 
soldiers,  which  was  collected  using  a  set  of  rating  scales  very  similar  to 
those  used  to  collect  the  present  data  (Campbell  &  Zook,  1990). 

HOS-Specific  Scales 

Parallel  to  what  reported  for  the  Army-wide  scales,  results  relative 
to  MOS-specific  rating  distributions  and  interrater  reliabilities  are 
descr i bed  be  low. 

Rating  distributions.  The  means  and  standard  deviations  of  the  MOS- 
specific  ratings  for  each  Batch  A  MOS  are  presented  in  Table  4.14,  Peer  and 
supervisor  rating  results  are  shown  separately.  In  general,  the  means  and 
standard  deviations  for  this  LVI  sample  are  exti'emely  similar  to  those 
o.btained  for  the  CVI  sample. 
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Table  -Kil 


Comparvson  of  LVI  and  CVI 
Peer /Supervisor  Ratings** 

Army-Wide  Factor  Analysis*  Results:  Pooled 

Factor  Lcadinos 

(Lvi/cvn 

Dimension 

1 

2 

3 

Technical  Knowledge/Ski 'i  1 

.30  /  .28 

.38  /  .30 

Leadership 

..34  /  .30 

.44  /  .37 

Effort 

■  56  / 

.47  /  .43 

.32  /  .26 

Self -Development 

i  ^ 

.42  /  .38 

.46  /  .38 

Maintain  if'  Equipment 

.50  /  ^ 

.41  /  .34 

.41  /  .35 

Following  Reguiations 

.39  /  .41 

.73  /  .69 

.31  /  .30 

5,e  1  f  -  Contro'i 

.1?  /  .22 

.05  /  ^3 

.20  /  .20 

Integr ity 

.44  /  .50 

.•-&6  /  .._59 

.30  /  .26 

Military  Appearance 

.31  /  .32 

.35  /  .32 

Jii  / 

Physical  Fitness 

.24  /  ,21 

.16  /  .15 

iii  /  ^ 

Percent  Common  Variance 

37.7  /  44.9 

36.6  /  32.7 

25.6  /  22.4 

Note.  Sample  size  is  7,919  for  LV!  and  8,642  tor  CVl. 

^  Pr-incipal  factor  analysis,  varimax  rotation. 

Computed  by  averaging  the  mean  peer  rating  and  the  mean  supervisor  rating. 
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Table  4.12 


Comparison  of  LVI  and  CVI  Army-Wide  Factor  Analysis^  Results:  Peer  Ratings 


Dimension 

Factor  Loadinas  (LVI/CVI) 

1 

2 

3 

Following  Regulations 

.68  / 

.65 

.31  /  .28 

.33  / 

.34 

Integrity 

■  61  / 

.55 

.32  /  .29 

.40  / 

.43 

Self-Control 

.58  / 

.57 

.22  /  .19 

.19  / 

.20 

Mi  1  itary  Appearance 

.35  / 

.31 

.57  /  ^ 

.25  / 

.28 

Physical  Fitness 

.18  / 

.15 

.52  / 

.25  / 

.22 

Self-Development 

.44  / 

.40 

.48  ./  .38 

•42  /  ^ 

Maintaining  Equipment 

.40  / 

.35 

.44  /  .39 

.39  /  , 

.43 

Effort 

.45  / 

.45 

.36  /  .29 

.58  /  , 

.•10 

Technical  Knowledge/Skill 

.33  / 

.30 

.40  /  .32 

.58  /  , 

J1 

Leadership 

.38  / 

.32 

.48  /  .40 

/  . 

.62 

Perce.  .ommon  Variance 

37.9  / 

35.1 

31.7  /  26.4 

30.6  /  38.5 

Note.  Sample  size  is  9,021  for  LVI  and  8,G35  for  CVI, 


^  Principal  factor  analysis,  varimax  rotation. 
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Table  4.13 


Comparison  of  LVI  and  CVI 
Ratings 

Army-Wide 

Factor 

‘  Analysis'*  Results:  Supervisor 

Dimension 

Factor  Loadinos 

fLVI/CVn 

1 

2 

3 

Leadership 

.67  / 

.68 

.31  /  .30 

.36  /  .34 

Technical  Knowledge/Skill 

.67  / 

.69 

.27  /  .26 

.32  /  .29 

Effort 

.66  / 

.70 

.44  /  .40 

.28  /  .25 

Self-Development 

^■1  / 

.55 

.34  /  .34 

.44  /  .39 

Maintaining  Equipment 

.50  / 

■  53 

.37  /  .32 

.40  /  .38 

Following  Regulations 

.41  / 

.42 

■  68  /  M 

.29  /  .30 

Integrity 

.44  / 

.49 

M  ^ 

.28  /  .29 

Self  Control 

.19  / 

.22 

■  61  /  ^ 

.22  /  .23 

Military  Appearance 

.33  / 

.32 

.35  /  .32 

. 54  /  . 55 

Physical  Fitness 

.24  / 

.19 

.18  /  .17 

.47  /  ^ 

Percent  Common  Variance 

41,6  /  45.9 

34.5  /  31.1 

24.0  /  23.0 

Note.  Sample  size  is  9,728  for  LVI  and  8,642  for  CVI. 
^  Principal  factor  analysis,  varimax  rotation. 


Interrater  reliability.  Interrater  reliability  results  for  the  MOS- 
specific  ratings  are  shown  in  Tables  4.15  and  4.16.  Table  4.15  presents  one- 
rater  reliabilities.  As  with  the  Army-wide  ratings,  peer  ratings  are  not  very 
reliable  at  the  one-rater  level  and  supervisor  ratings  are  considerably  more 
reliable.  Table  4.16  shows  that  the  mean  dimension  level  peer  and  supervisor 
ratings  are  considerably  more  reliable,  but  not  on  average  as  reliable  as  the 
Army-wide  mean  dimension  ratings. 
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Table  4.14 


LVI  M03-Specific  Ratings:  Means  Across  All  Rating  Dimensions,  by  HOS 


*  *  '’a- 

MOS 

* 

_ LLSA 

IIB 

13B 

19F 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

Number  of 

Rating  Dimensions 

13 

11 

9 

9 

7 

12 

9 

11 

10 

12 

Peer  Ratings 

Mean 

4.70 

4.69 

4.85 

4.76 

4.80 

4.75 

5.04 

4.89 

4.81 

4.70 

Standard  Deviation 

1.03 

1.04 

.96 

.96 

1.12 

1.15 

1.13 

1.04 

1.04 

.97 

Average  Number 
of  Ratees 

823 

799 

232 

631 

327 

530 

307 

568 

685 

424 

Supervisor  Ratings 

Mean 

4.84 

4.70 

4.65 

4.73 

4.70 

4.30 

4.72 

4.70 

4.65 

4.62 

Standard  Deviation 

1.13 

1.27 

1.10 

1.07 

1.22 

1.28 

1.23 

1.19 

1.20 

1.05 

Average  Number 
of  Ratees 

821 

744 

216 

620 

414 

669 

561 

601 

663 

410 
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is  the  mean  number  of  ratings  per  ratee. 


Conclusions 


For  both  the  Army-wide  and  MOS-specific  rating  scales,  the  mean, 
variability,  and  reliability  of  the  peer,  supervisor,  and  pooled  peer/super¬ 
visor  ratings  apppar  quite  acceptable  and  are  comparable  to  what  was  found  in 
the  CVI  research.  Factor  analyses  of  the  Army-wide  ratings  showed  that  the 
three-factor  CVI  solution  is  replicated  in  the  present  data.  Accordingly,  the 
three  composites  shown  in  Table  4.17  are  proposed  as  the  basic  scores  for  the 
Army-wide  rating  data.  As  in  CVI  research,  unit  weighting  of  each  dimension 
is  recommended  when  computing  scores  for  each  rating  composite.  Definitions 
for  the  composites  are  also  presented  in  Table  4.17. 


Table  4.17 


Composition  and  Definition  of  Proposed  LVI  Army-Wide  Rating  Composites 


Factor  Name  and  Definition 

Percent  Common 
Variance  Accounted 
For  by  Relevant 
Factor^  (LVI/CVI) 

Dimensions  Included 

1.  Technical  Skills  and  Job 
Effort: 

Exerting  effort  over  the  full 
range  of  job  tasks;  engaging  in 
training  or  other  development 
activities  to  increase 
proficiency;  persevering  under 
dangerous  or  adverse  conditions; 
and  demonstrating  leadership  and 
support  toward  peers. 

37.8/44.9 

Technical  Knowledge/Sk i 1 
Leadership 

Effort 

Self-development 
Maintaining  Equipment 

2.  Personal  Discipline: 

Adhering  to  Army  ruler,  and 
regulations;  exercising  self- 
control;  demonstrati.;  integrity 
in  day-to-day  behavior;  and  not 
causing  disciplinary  problems. 

36.6/32.7 

Following  Regulations 
Self-control 

Integrity 

3.  Physical  Fitness/Military 
Bearing: 

25.6/22.4 

Mi  1 itary  Bearing 

Physical  Fitness 

Maintaining  an  appropriate- 
military  appearance  and  bearing 
and  staying  in  good  physical 
cond ition . 

^  Factor  analysis  of  pooled  peer/supervisor  ratings. 
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The  interrater  reliabilities  of  these  three  unit-weighted  composites  are 
shown  in  Table  4.18.  As  with  the  dimension-level  ratings,  the  pooled  ratings 
are  the  most  reliable.  The  pooled  peer  and  supervisor  ratings  may  also  be 
more  valid  in  the  sense  that  they  draw  on  both  peer  and  supervisor 
perspectives.  It  has  been  argued  (Borman,  1974;  Campbell,  Dunnette,  Lawler,  & 
Weick,  1970)  that  typically  in  organizations  peers  and  supervisors  each  have 
relevant  and  important  performance  information  on  ratees  but  that  these  two 
rating  sources  may  have  di'.'ferent  information  because  of  different  roles  and 
opportunity  to  observe  performance-related  behavior.  Thus,  we  would  expect 
that  peer  and  supervisor  ratings,  taken  together,  should  provide  more  valid 
performance  information  on  ratees  than  either  of  the  rating  sources 
individual ly. 


Table  4.18 

One-Rater  and  k-Rater®  Reliabilities  of  Proposed  LVI  /^rmy-Wide  Rating 


Composites 

Technical  Personal 

Skill/Effort  Discipline 

Fitness/ 

Bearing 

Peer  Ratings 


One-Rater 

.32 

.29 

.33 

J^-Rater 

.57 

.54 

.58 

Average  Ratings  per  Ratee 

2.86 

2.86 

2.86 

Supervisor  Ratings 

One-Rater 

.47 

.44 

.49 

]^-Rater 

.61 

.58 

.64 

Average  Ratings  per  Ratee 

1.81 

1.81 

1.81 

Pooled  Peer/Supervisor  Ratings*’ 

Coefficient  Alpha 

.61 

.60 

.65 

^  is  the  mean  number  of  ratings  per 

Computed  by  averaging  the  mean  peer 

ratee. 
rating  anu 

the  mean  supervisor 

rating. 
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Evidence  that  peer  and  supervisor  raters  are  not  parallel  measures  of 
the  same  thing  is  contained  in  the  values  for  coefficient  alpha  when  the 
average  peer  and  average  supervisor  ratings  are  used  as  the  two  components  of 
a  composite.  The  reliability  of  the  composite  is  not  as  high  as  would  be 
expected  if  they  were  parallel  measures.  In  general,  the  intercorrelations 
among  peers  and  among  supervisors  are  higher  than  the  intercorrelations 
between  peers  and  supervisors.  At  the  extreme,  coefficient  alpha  of  the 
composite  of  average  peer  plus  average  supervisor  ratings  would  be  zero  if  the 
intercorrelations  between  peers  and  supervisors  were  all  zero. 

Correlations  among  the  three  Army-wide  unit-weighted  composites  are 
presented  in  Table  4.19.  Although  some  of  these  correlations  are  quite  high, 
on  the  basis  of  our  experience  in  CVI  there  should  be  sufficient  di'^ferentia- 
tion  between  these  LVI  composites  to  provide  multidimensional  performance 
information . 


Table  4.19 


Intercorrelations  Among  Proposed  LVI  Rating  Composites 


Technical 

Skill/Effort 

Personal 

Discipline 

Based  on  Peer  Ratings  Only^ 

Personal  Discipline 

.37 

F i tness/Bearing 

.66 

.59 

Based  on  Supervisor  Ratings  Only^ 

Personal  Discipline 

.86 

F i tness/Bearing 

.  63 

.59 

Based  on  Pooled  Peer/Supervisor  Ratings^ 

Personal  Discipline 

.89 

Fitness/Bearing 

.65 

.60 

a 

b 

c 


N  =  9,114  - 
N  =  9,932  - 
Computed  by 
N  =  a, 158  - 


9,121. 

9,937. 

averaging 

8,164. 


the  mean  peer  rating  and  the 


!ean  supervisor  rating; 
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For  the  MOS-specific  ratings,  the  Interrater  reliabilities  of  the  unit- 
weighted  composites  of  rtii  dimensions  are  shown,  by  MOS,  in  Table  4.20.  The 
mean  supervisor  ratings  are  generally  more  reliable  than  the  pooled  peer  and 
supervisor  ratings  for  these  MOS-specific  rating  scales.  However,  as  with  the 
Army-wide  ratings,  the  pooled  ratings  could  be  seen  as  containing  more 
information  about  job  performance  than  the  supervisor  ratings  alone.  This 
would  be  especially  applicable  lor  MOS  31C  where  coefficient  alpha  is  lower 
than  the  reliability  for  either  peers  or  supervisors.  The  composite  is  mult-’- 
dimensional . 


Table  4.20 

One-Rater  and  k-Rater*  Reliabilities  of  Proposed  LVI  HOS-Specific  Rating 
Composites _ _ _ 


MOS 

IIB 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

Peer  Ratings 

One-Rater 

.27 

.18 

.15 

.20 

.35 

.25 

.15 

.22 

.27 

.26 

k.-Rater 

,52 

.40 

.32 

.44 

.  54 

.41 

.24 

.46 

.52 

.54 

Average  Ratings 

Per  Ratee 

2.89 

2.93 

2.57 

3.21 

2.23 

2.03 

1.71 

3.00 

3.03 

3.34 

Supervisor  Ratings 

One-Rater 

.40 

.31 

.38 

.28 

.39 

.46 

.44 

.42 

.40 

.37 

l^-Rater 

.  56 

.45 

.54 

.44 

.  53 

.59 

.57 

.57 

.53 

.52 

Average  Ratings 

Per  Ratee 

1.92 

1 .82 

1.89 

2.05 

1.73 

1.68 

1.66 

1.84 

1.71 

1.82 

Pooled  Peer/ 
Supervisor  Ratings’^ 

Coefficient  Alpha 

.51 

.53 

.42 

.51 

.29 

.57 

.37 

.50 

.51 

.61 

^  is  the  mean  number  of  ratings  per  ratee. 

^  Computed  by  averaging  the  mean  peer  rating  and  the  mean  supervisor  rating. 
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In  sum,  the  Army-wide  ratings  provide  a  reliable  and  interpretable 
multidimensional  depiction  of  three  different  performance  areas.  The  MOS- 
specific  ratings  provide  a  single,  reliable  job  performance  composite  that 
also  represents  a  significant  component  of  total  performance. 


DEVELOPMENT  OF  BASIC  SCORES  FOR  HANDS-ON  AND 
JOB  KNOWLEDGE  PERFORMANCE  MEASURES 

This  section  outlines  the  procedures  used  to  formulate  the  basic 
criterion  scores  for  the  first-tour  Hands-On  and  Job  Knowledge  measures.  The 
analyses  represent  a  replication  of  the  CVI  efforts  described  in  the  Project  A 
Annual  Report  for  1986  (Campbell,  1988).  There  were  two  specific  objectives: 

(a)  to  edit  the  data  in  accordance  with  previously  established  procedures  so 
as  to  maximize  available  sample  sizes  without  compromising  data  quality  and 

(b)  to  aggregate  the  individual  item,  task,  or  scale  scores  into  a  set  of  more 
manageable  basic  scores  that  preserve  the  original  information  and  that  can  be 
used  ir-.  constructing  a  model  of  job  performance  (see  Chapter  6). 


Definition  of  Terms 

Certain  terms  used  throughout  this  section  have  meaning  that  is  specific 
to  this  subject  matter.  As  an  aid  to  understanding,  the  more  critical  terms 
are  defined  below. 

Task ;  A  discrete  set  of  behaviors  performed  to  accomplish  a  single  job 
requirement;  includes  a  situation  with  initiating  cues  and  conditions, 
the  steps  or  activities  that  are  to  be  performed,  and  the  task  standards 
that  signal  successful  completion. 

Common  Task:  A  task  drawn  from  the  Soldier's  Manual  of  Common  Tasks. 
Skill  Level  1  (STP  21-1-SMCT,  October  1985)  or  from  the  Soldier's  Manual 
of  Common  Tasks,  Skill  Level  2/3/A  (STP  21-24-SMCT,  Draft,  January 
1987).  These  are  basic  soldiering  tasks  (first  aid,  personal  weapons, 
map  reading,  etc.)  that  are  required  of  all  soldiers,  regardless  of  MOS. 
Soldiers  are  responsible  for  performance  of  all  common  tasks  at  their 
own  skill  level;  in  some  cases,  the  task  selection  panel  also  chose 
higher  skill  level  tasks  for  testing. 

MQS-specific  Task;  A  task  drawn  from  the  Soldier's  Manual  for  a  given 
MOS.  These  tasks  are  central  to  the  job  of  the  soldiers  in  the  MOS  and 
are  typically  unique  to  the  MOS.  As  with  common  tasks,  soldiers  are 
responsible  for  tasks  at  their  own  skill  levels,  and  the  task  selection 
panel  may  also  have  chosen  higher  skill  level  tasks  to  represent  the 
job. 

Basic  Tasks:  Tasks  that  represent  common  soldiering  skills  but  are  not 
necessarily  mar’datLl  for  every  MOS.  Most  of  the  Basic  Tasks  are  Common 
Tasks  (i.e.,  drown  from  the  Soldier's  Manuals  of  Common  Tasks).  Many 
are  required  only  by  one  or  very  few  MOS,  yet  do  not  represent  MOS- 
specific  specialized  knowledges  or  skills. 
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Technical  Tasks:  Tasks  that  represent  specialized  knowledges  or  skills 
that  are  peculiar  to  specific  MOS  in  both  the  requirement  to  perform  and 
the  underlying  abilities  contributing  to  proficiency. 

Task  test:  A  set  of  performance  steps  or  written  items  that  are  focused 
on  a  single  task.  Hands-on  test  performance  steps  are  scored  GO  or  NO- 
GO  by  a  trained  scorer.  Job  knowledge  test  written  items  are  multiple 
choice,  with  one  correct  answer  per  item,  and  are  scored  as  Pass  or 
Fail. 

Hands-on  component;  The  full  set  of  hands-on  performance  (job  sample) 
task  tests  for  an  MOS;  comprises  14-17  task  tests. 

Job  knowledge  component:  The  full  set  of  job  knowledge  (written)  task 
tests  for  an  MOS;  comprises  28-30  task  tests. 

Track :  A  separate  version  of  a  hands-on  or  knowledge  test  prepared  to 
accommodate  different  types  of  equipment  that  may  be  used  in  performing 
the  same  task. 

First  tour:  A  term  used  within  Project  A  and  the  Career  Force  project 
to  designate  soldiers  who  have  been  in  service  for  1  to  3  years. 


MOS  Task-Specific  Performance  Content 

The  procedures  that  were  used  to  select  tasks  and  to  develop  task  tests 
for  each  of  the  nine  MOS  are  described  in  the  Project  A  Annual  Report  for  1985 
(Campbell,  1987).  Before  the  instruments  that  had  been  developed  for  the 
Concurrent  Validation  criterion  measurement  in  1985  could  be  used  again, 
technical  currency  reviews  were  necessary.  Each  hands-on  and  job  knowledge 
component  was  subjected  to  rigorous  review  against  existing  Army  doctrinal 
training  materials  by  project  staff,  and  revisions  were  made  to  items  and 
Supporting  graphics  and  handouts  as  necessary.  All  revisions  were  evaluated 
by  the  MOS  proponent  agencies  to  ensure  the  technical  currency  of  the  tasks 
ana  task  steps.  Their  evaluation  led  to  the  decision  to  drop  some  st''ps  or 
items  and  task  tests  because  they  addressed  performance  requirements  that  were 
no  longer  doctrinally  correct.  A  list  of  the  tasks  comprising  the  hands-on 
and  job  knowledge  test  components  for  each  MOS  is  presented  in  Appendix  A. 

As  this  phase  of  the  project  began,  a  tenth  MOS,  19K,  the  Ml  Abrams 
Armor  Crewman,  was  added  to  the  Batch  A  cluster  of  MOS  to  cover  the  Army's 
transition  from  the  M60-series  tank  (with  MOS  19E)  to  the  Ml-series  tank.  The 
steps  in  selecting  tasks  and  developing  tests  for  this  MOS  were  tne  same  as 
those  that  had  been  followed  for  the  other  MOS. 

Data  Edits  to  Control  Variance 

As  with  the  CVI  analyses,  there  were  five  known  sources  of  extraneous 
variance  that  v”  wished  to  control:  track  differences,  missing  data,  site 
differences  '  i ’•erences  in  the  number  of  test  steps/items  across  tasks,  and 
items  with  rr„i,  .,al  psychometric  characteristics.  The  procedures  that  were 
developed  to  ni.niinize  t^e  effects  of  these  sources  of  variance  in  the  CVI 
analyses  were,  for  thr  most  part,  also  followed  here. 
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For  hands-on  tests,  tracked  versions  were  prepared  as  necessary  to 
accommodate  the  requirement  to  test  using  different  types  of  equipment  on  the 
same  task.  These  tracks  are  intended  to  be  parallel  versions  of  the  same 
test.  In  some  cases,  equipment  variations  required  only  minor  changes  in  a 
few  hands-on  steps,  or  the  omission  of  a  few  steps.  In  other  cases,  althougfi 
equipment  and  procedures  were  dissimilar  across  the  tracks,  the  behavioral 
requirements  were  judged  to  be  the  same  and  provided  comparable  representation 
of  job  content.  As  with  the  CVI  analyses,  we  examined  the  LVI  results  for 
evidence  of  level  and  dispersion  differences  across  tracks.  No  anomalous 
differences  were  found.  Consequently  the  percentage  of  steps  passed  in  a  task 
was  used  as  the  task  score.  This  transformation  corrects  for  the  variation  in 
number  of  steps  performed,  while  at  the  same  time  preserving  information  about 
individual  differences. 

Similarly,  scores  on  tracked  task  tests  in  the  job  knowledge  measures 
were  scored  on  percentage  correct  to  account  for  different  numbers  of  items, 
but  were  not  otherwise  adjusted. 

The  second  adjustment  was  for  missing  data.  For  some  tasks,  the 
difficulties  associated  with  obtaining  equipment  for  certain  specialized  tests 
at  different  sites  precluded  hands-on  testing,  either  entirely  or  for  segments 
of  the  task.  Data  could  also  be  missing  for  one  of  two  other  reasons:  Either 
the  scorer  failed  to  observe  a  step,  or  the  scorer  failed  to  record  the 
observation  as  either  GO  or  NO-GO.  In  either  event,  the  fact  that  the 
observation  was  missing  was  irrelevant  to  the  soldier's  performance.  In  the 
job  knowledge  tests,  there  were  also  two  likely  reasons  for  missing  data: 
Either  the  soldier  skipped  an  item  (did  not  record  a  response  on  the  answer 
sheet),  or  the  soldier  did  not  get  to  one  or  more  items  at  the  end  of  the  test 
booklet.  The  methods  used  to  adjust  for  missing  data  are  discussed  in  greater 
detail  in  the  next  chapter. 

The  third  source  of  extraneous  variance  is  an  issue  only  with  the  hands- 
on  tests.  There  was  concern  that  the  hands-on  test  scores  might  reflect  site 
differences  in  the  way  the  measures  were  administered.  Type  of  testing 
facility,  condition  of  equipment,  local  training  emphasis  and  operating 
procedures,  and  *'eather  and  terrain  conditions  could  interfere  with 
standardization  of  conditions.  Therefore,  as  with  the  CVI  results,  hands-on 
test  scores  were  standardized  by  site  at  the  task  level  to  control  for  the 
differences. 

A  fourth  source  of  irrelevant  variance  consisted  of  the  differences  in 
the  number  of  items  or  steps  used  to  cover  specific  task  tests.  The  number  of 
items  was  not  necessarily  related  to  the  Cv^.T.plcxity  of  the  task,  and  we  did 
not  want  some  tasks  to  be  given  more  weight  than  others.  For  hands-on  tests, 
where  the  number  of  steps  per  task  varied  widely,  task  scores  were  computed  as 
percent-GO,  and  these  task-level  scores  were  used  in  subsequent  analyses.  For 
job  knowledge  tests,  where  the  number  of  items  per  task  (or  per  task  cluster, 
in  subsequent  analyses)  did  not  vary  as  much,  a  percent-correct  score  was 
computed  at  each  level  (task,  cluster,  etc.).  Table  4.21  shows  the  overall 
number  of  steps  in  the  hands-on  component  for  each  MOS  and  the  range  of  steps 
per  task  test.  Table  4.22  shows  the  overall  number  of  items  in  the  job 
knowledge  component  for  each  MOS  and  the  range  of  items  per  task  test,  as  well 
as  the  nature  of  revisions  for  marginal  items  (described  in  the  next 
paragraph )  . 
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Table  4.21 


Number  of  LVI  Hands-On  Tests  and  Steps 


MOS 

m 

Steps 

Range 

Average  Steps 

Per  Task 

IIB 

Infantryman® 

13-14 

206-238 

4-43 

17.2-18.3 

138 

Cannon  Crewman^ 

16 

301-308 

5-36 

18.8-19.2 

19E 

M60  Armor  Crewman 

15 

202 

6-31 

12.5 

19K 

Ml  Armor  Crewman^ 

14 

214-231 

6-36 

15.3-16.5 

31C 

Single  Channel  Radio  Operator*^ 

15 

353-361 

8-72 

23.5-24.0 

638 

Light-Wheel  Vehicle  Mechanic 

14 

152 

4-20 

10.9 

71L 

Administrative  Specialist® 

14 

165 

2-43 

12.7 

88M 

Motor  Transport  Operator 

15 

280 

3-43 

18.7 

91A 

Medical  Specialist 

14 

222 

7-39 

15.9 

95B 

Military  Pol ice^ 

13 

206-233 

4-39 

17.2-19.4 

^  For  IIB,  the  number  of  tasks  does  not  include  Engage  Targets  with  M16-Series 
Rifle,  tested  on  the  MACS.  Most  soldiers  were  not  tested  on  the  task,  Set 
Headspace  and  Timing  on  .50  Caliber  Machinegun.  The  task,  Conduct 
Surveillance  Without  the  Use  of  Electronic  Devices,  is  counted,  but  it  has  a 
single  continuously  scored  product  score  rather  than  steps;  th  task  is  not 
included  in  the  Average  Steps  Per  Task  figure. 

^  For  138,  six  tasks  were  tested  in  three  tracks,  with  a  slightly  different 
number  of  steps  per  track. 

^  For  19K,  one  task  was  tested  in  two  tracks,  with  a  slightly  different  number 
of  steps  oer  track. 

For  31C,  one  task  was  tested  in  two  tracks,  with  a  slightly  different  number 
of  steps  per  track. 

®  For  71L,  the  task.  Type  Straight  Copy,  is  counted,  but  it  has  a  single 
continuously  scored  product  score  rather  than  steps.  The  task  is  not 
included  in  the  Average  Steps  Per  Task  figure. 

For  958,  the  number  of  tasks  does  not  include  Engage  Targets  with  M16-Series 
Rifle,  tested  on  the  MACS.  One  task  vs  tested  in  three  tracks,  with  a 
slightly  different  number  of  steps  pe.  track. 
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Finally,  the  job  knowledge  test  data  were  adjusted  for  marginal  items  -- 
those  items  that  appeared  to  be  no  longer  content  valid.  Because  of  changes 
in  equipment  and  changes  in  the  proscribed  steps  in  performance  between  the 
CVI  testing  and  the  LVI  testing,  not  all  test  items  were  correct  when  the 
tests  were  administered  --  this  despite  rigorous  currency  review  and  careful 
proponent  agency  examination.  In  some  cases,  another  of  the  alternative 
answers  was  now  correct,  and  simple  rekeying  corrected  the  problem;  in  other 
cases,  more  than  one  response  was  now  correct,  and  the  item  was  double-keyed. 
In  still  other  cases,  no  correct  answer  was  now  included  in  the  list  of 
responses,  and  those  items  were  zero-weighted  (i.e.,  dropped).  Table  4.22 
shows  the  extent  of  the  revisions  or  deletions. 


Table  4.22 

Number  of  LVI  Job  Knowledge  Tests  and  Items,  and  Revisions  to  Eliminate 
Marginal  Items _ 


MOS 

Number 

of 

Tasks 

Items 

Revised/ 

Dropped 

Total 
Number 
of  Items 

Range 

Average 
Items  per 
Task 

IIB 

I  nfantryman^ 

28-29 

6/1 

175-186 

2-12 

6. 2-6.4 

13B 

Cannon  Crewman^ 

30 

7/2 

177-180 

2-i6 

5. 9-6.0 

19E 

M60  Armor  Crewman 

30 

6/2 

191 

4-12 

6.4 

19K 

Ml  Armor  Crewman* 

30 

9/7 

172-173 

3-12 

5. 7-5.8 

31C 

Single  Channel  Radio  Operator 

30 

6/2 

204 

3-12 

6.8 

63B 

Light-Wheel  Vehicle  Mechanic 

29 

3/0 

188 

3-11 

6.5 

71L 

Administrative  Specialist 

23 

9/1 

121 

3-9 

5.3 

88M 

Motor  Transport  Operator 

28 

1/0 

163 

2-11 

5.8 

91A 

Medical  Specialist 

^9 

6/3 

214 

3-12 

7.4 

95B 

Mi  1  itary  Pol  ice'^ 

30 

18/5 

198-200 

2-13 

6.6-6. 7 

^  For  IIB,  most  soldiers  were  not  tested  on  the  task,  Set  Headspace  and  Timing 
on  the  .50  Caliber  Machinegun. 


^  For  13B,  14  tasks  were  tested  in  three  tracks,  with  a  slightly  different 
number  of  items  per  track. 

For  19K,  one  task  was  tested  in  two  tracks,  with  a  slightly  oitferent  number 
of  items  per  track. 

For  95B,  one  task  was  tested  in  two  tracks,  with  a  slightly  different  number 
of  items  per  track. 
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Construction  of  Basic  Criterion  Scores 


As  the  first  step  in  replicating  the  CVI  procedures  for  constructing  the 
basic  scores,  tasl^s  were  clustered  into  Functional  Categories  (as  described  in 
the  Project  A  Annual  Report  for  1986  (Campbell,  1988)).  The  Functional 
Category  rules  developed  for  the  CVI  define  10  across-MOS  categories,  plus  one 
to  five  MOS-specific  Technical  Categories  per  MOS  (except  IIB,  where  all  tasks 
fit  into  the  10  across-MOS  categories). 

The  Functional  Category  scores  for  job  knowledge  tests  were  computed  as 
the  percentage  of  items  correct  for  all  tasks  assigned  to  the  category.  For 
hands-on  tests,  the  Functional  Category  score  was  the  mean  of  the  percentage 
of  steps  performed  correctly  for  each  hands-on  task  test  in  the  category. 

Following  the  procedures  developed  with  the  CVI  data,  tasks  were  also 
sorted  into  six  higher-level  groups  referred  to  as  Task  Factors  (Communica¬ 
tion,  Vehicles,  Basic  Techniques,  Identify  Targets,  Technical,  and  Safety/ 
Survival  (CVBITS)).  Tasks  were  also  combined  into  just  two  groups:  Basic 
(i.e.,  Army-wide)  and  Technical  (i.e.,  MOS-specific). 

In  general,  the  grouping  schemes  are  hierarchical:  Tasks  (the  lowest 
level)  are  placed  in  Functional  Categories,  the  Functional  Categories  (level 
two)  are  aggregated  to  form  the  six  Task  Factors  (level  three),  and  Task 
Factors  are  then  aggregated  to  form  the  two  Task  Constructs  (level  four),  as 
diagrammed  in  Figure  4.1. 

There  were  three  exceptions  to  the  hierarchical  scheme.  One  exception 
is  the  IIB  Infantryman  MOS,  in  wtiirh  all  tasks  are  assigned  to  the  across-MOS 
Functional  Categories  (level  two)  and  to  the  across-MOS  Task  Factors  (level 
three).  At  the  Task  Construct  level  (level  4),  however,  the  IIB  tasks  are  all 
placed  under  the  Technical  heading,  rather  than  under  the  Basic  heading.  The 
second  exception  concerns  the  tasks  involving  vehicle  maintenance  and 
operation:  for  the  633  Light-Wheel  Vehicle  Mechanic,  tasks  in  the  MOS-specific 
Functional  Category  labeled  Vehicle  Operations  and  Recovery  are  not 
subsequently  placed  into  the  Technical  Task  Factor,  but  rather  into  the 
Vehicles  Task  Factor  and  thence  into  the  Basic  Task  Construct.  The  third 
exception  is  in  the  case  of  the  83M  Mo  .or  Transport  Operator,  where  all  tasks 
fell  into  the  across-MOS  Functional  Categories;  as  a  result  the  88M  has  no 
Technical  Task  Construct,  These  designations  are  the  convention  that  was  used 
for  the  current  analyse:;.  For  reasons  that  will  be  described  in  Chapter  7  a 
slightly  different  convention  was  used  for  the  LVI  performance  modeling 
analyses . 

Not  every  MOS  had  representation  in  every  Functional  Category,  nor  in 
every  Task  Factor.  The  task  lists  in  Appendix  A  identify  the  Functional 
Category  and  Task  Factor  for  every  task.  With  two  exceptions  ^ the  IIB  and 
88M,  as  described  above),  every  MOS  has  tasks  in  both  the  Basic  and  Technical 
Task  Constructs.  For  job  knowledge  tests,  the  scores  for  each  level  are 
formed  by  computing  the  proportion  of  items  passed  acrosi  all  tasks  in  the 
grouping.  For  hands-on  tests,  the  score  for  each  grouping  is  the  mean  of  the 
task  (proportion  passed)  scores. 
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Tasks  (Exasples) 


Functional 

categories 


Task  Factors 


Task  constructs 


BASIC 


Hcvitzer  prefire  MDS-Specific 
Fire  tank  main  gun  Categories 
Install  radio  set 
Troubleshoot  brakes 
Type  military  orders 
Give  injection 
c^erate  discount  point 


Technical 


TEOiNICAL 


NCT£:  Exceptions  to  the  hierarchical  structure  aure  shown  in  Appendix  A. 


Figure  4.1.  Hierarchical  relationships  among  Functional 

Categories,  Task  Factors,  and  Task  Constructs. 


Confirmatory  Analyses  of  Alternative  Definitions 
of  Basic  Hands-On  and  Job  Knowledge  Scores 


As  noted  above,  the  Functional  Categories  were  constructed  via  expert 
judgment.  The  number  of  variables  was  reduced  further  by  a  series  of 
exploratory  and  confirmatory  analyses.  The  result  of  the  first  round  of 
exploratory  analyses  in  CVI  was  the  set  of  six  Task  Factors  (Campbell,  1988). 
When  scores  constructed  on  these  variables  were  subjected  to  another  round  of 
empirical  factor  analysis  with  other  criterion  variables  (from  various  rating 
scales  and  administrative  records),  the  hands-on  and  job  knowledge  test 
components  each  split  between  two  higher  order  factors,  with  the  Basic  (non- 
MOS-specif ic)  Task  Factor  scores,  for  both  hands-on  and  job  knowledge  tests, 
loading  on  the  factor  labeled  General  Soldiering  Proficiency,  and  the 
Technical  (MOS-specif ic)  Task  Factor  scores,  for  both  hands-on  and  job 
knowledge  tests,  loading  on  the  factor  labeled  Core  Technical  Proficiency. 

For  the  LVI  data,  confirmatory  factor  analyses  were  conducted  to  assess 
the  fit  of  alternative  levels  of  score  aggregation.  These  analyses  served  two 
purposes:  They  were  used  to  assess  the  relative  merits  of  each  model  and  to 
corroborate  the  CVI  decision  to  use  the  six  task  factor  scores  (CVBITS).  The 
analysis  required  the  computation  of  separate  tests  of  goodness  of  fit  for 
hands-on  and  job  knowledge  test  data,  for  each  of  the  10  MOS,  on  each  of  three 
competing  models.  The  three  models  to  be  tested  were:  a  one-factor  model, 
postulating  the  existence  of  a  single  factor  in  the  data;  a  two-factor  model, 
proposing  the  Basic  and  the  Technical  Task  Constructs;  and  three-to-six-factor 
model  (the  number  of  factors  varying  among  MOS  and  component),  using  the  Task 
Factors.  These  were  done  using  the  LISREL  confirmatory  factor  analysis 
program  (Joreskog  &  Sorbom,  1981).  The  approach  parallels  the  confirmatory 
analyses  conducted  in  the  CVI  model-building  process. 

In  conducting  a  confirmatory  factor  analysis  with  LISREL,  it  is 
necessary  to  specify  the  structure  of  three  different  parameter  matrixes: 
Lambda-Y,  the  hypothesized  factor  structure  matrix  (a  matrix  of  regression 
coefficients  for  predicting  the  observed  variables  from  the  underlying  latent 
constructs);  Theta-Epsilon,  the  matix  of  uniqueness  or  error  commponents  (and 
intercorrelations);  and  Psi,  the  matrix  of  covariances  among  the  factor-s.  For 
each  analysis,  the  diagonal  elements  of  Psi,  the  factor  covariance  matrix, 
were  set  to  1.0,  forcing  a  "standardized"  solution.  Off-diagonal  elements  in 
Psi  thus  represented  the  correlations  among  the  factors. 

As  a  product  of  the  analysis,  LISREL  computes  a  goodness-of-f it  index 
based  on  a  comparison  of  the  actual  correlations  among  the  observed  variables 
and  the  correlations  estimated  from  Lambda-Y,  Theta-Epsilon,  and  Psi.  The 
goodness  of  fit  iS  distributed  as  chi-square,  with  degrees  of  freedom 
dependent  on  the  number  of  observed  variables  and  the  number  of  parameters 
estimated.  The  expected  value  of  chi-square  is  equal  to  the  degrees  of 
freedom;  a  significant  chi-square  is  a  sign  that  the  model  does  not  fit  the 
correlations  among  the  observed  variables. 

Table  4.23  shows  the  value  of  chi-square  for  each  MOS  and  for  each  of 
the  three  alternative  criterion  score  model  structures  (the  single  criteron 
factor,  the  two  Task  Constructs,  and  the  three  to  six  Task  Factors),  for  ♦■he 
Job  Knowledge  Test  data.  Table  4,24  shows  the  same  information  for  the  h.-  ds- 
On  Test  data. 
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Table  4.23 


Goodness-of-FIt  Tests  for  Three  Models  for  LVI  Job  Knowledge  Tests  for  10  HOS 


MOS 

Model* 

Chi-Square 

df 

; 

( 

P 

Adjusted 
Soodness 
of  Fit 

Root  Mean 
Square 
Residual 

N 

IIB 

Infantryman^ 

One  Factor 

120.07 

20 

.000 

.939 

.048 

890 

SBC  I 

107.04 

18 

.000 

.940 

.044 

136 

Cannon  Crewman 

One  Factor 

106.52 

35 

.000 

.959 

.035 

809 

Two  Factor 

66.45 

34 

.001 

.973 

.028 

SBCIT 

50.51 

29 

.008 

.976 

.023 

19E 

M60  Armor  Crewman 

One  Factor 

52.43 

35 

.029 

.933 

.051 

248 

Two  Factor 

52.43 

35 

.029 

,933 

.051 

SBCIT 

33.52 

29 

.257 

.949 

.039 

19K 

Ml  Armor  Crewman 

One  Factor 

71.37 

35 

.000 

.972 

.030 

824 

Two  Factor 

66.05 

34 

.001 

.974 

.029 

SBCIT 

58.22 

29 

.001 

.973 

.027 

31C 

Single  Channel 

One  Factor 

159.45 

77 

.000 

.938 

.045 

528 

Radio  Operator 

Two  Factor 

138.03 

76 

.000 

.945 

.039 

SBCIVT 

110.94 

68 

.001 

.951 

.035 

638 

Light-Wheel 

One  Factor 

102.16 

44 

.000 

.961 

.038 

716 

Vehicle  Mechanic 

Two  Factor 

97.31 

43 

.000 

.962 

.036 

SBVT 

78.85 

40 

.000 

.967 

.031 

71L 

Administrative 

One  Factor 

78.64 

35 

.000 

.963 

.043 

671 

Special ist 

Two  Factor 

48.91 

34 

.047 

.976 

.033 

SBT 

46.76 

32 

.045 

.976 

.033 

88M 

Motor  Transport 

One  Factor 

32.65 

20 

.037 

.978 

.028 

674 

Operator'" 

SBIV 

28.78 

18 

.051 

.979 

.026 

91A 

Medical  Specialist 

One  Factor 

69.99 

27 

.000 

.967 

.035 

793 

Two  Factor 

66.22 

26 

.000 

.968 

.034 

SBIVT 

44.04 

21 

.002 

.974 

.022 

95B 

Mi  1 itary  Police 

One  Factor 

97.59 

44 

.000 

.940 

.048 

446 

Two  Factor 

90.39 

43 

.000 

.943 

.045 

SBCIVT 

65.25 

35 

.001 

.950 

.039 

The  one-factor  node)  tests  the  fit  of  a  single  factor  for  all  of  the  vor'ance  in  Job  Knowledge  Test 
scores.  The  two-factor  mooel  proposes  a  Basic  {across  MOS)  factor  and  a  Technical  (HOS-specif ic)  factor. 
The  third  model  factors  are  S  •  Siiety/'Surviva  1 .  8  •  Basic  Scldiering,  C  -  Coimtinicat ions ,  I  -  Identify 
Targets.  V  -  Vehicles,  and  7  -  Technii.al. 

^  MOS  IIB  has  no  Basic  Task  Construct. 

'  MOS  S8M  has  no  Technical  Task  Construct. 
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Table  4.24 


Goodness-of-Fit  Tests  for  Three  Models  for  LVI  Hands-On  Tests  for  10  HOS 


HOS 

Model*  Chi-Square 

df 

P 

Adjusted 
Goodness 
of  Fit 

Root  Mean 
Sguare 
Residua  1 

N 

118  Infantryman^ 

One  Factor 

13.27 

9 

.151 

.988 

.024 

890 

SBC 

9.41 

8 

.309 

.991 

.020 

13B  Cannon  Crewman 

One  Factor 

65.81 

20 

.000 

.962 

.050 

773 

Two  Factor 

55.02 

19 

.000 

.966 

.045 

SBCT 

38.55 

16 

.001 

.972 

.036 

19E  M60  Armor  Crewman 

One  Factor 

15.51 

14 

.344 

.963 

,049 

243 

Two  Factor 

15.51 

13 

.277 

.961 

.049 

SBCT 

8.71 

10 

.560 

.971 

.038 

19K  Ml  Armor  Crewman 

One  Factor 

61.89 

20 

.000 

.963 

.051 

749 

Two  Factor 

61.89 

19 

.000 

.961 

.051 

SBCT 

53.59 

16 

.000 

.960 

.045 

31C  Single  Channel 

One  Factor 

110.05 

35 

.000 

.922 

.083 

446 

Radio  Operator 

Two  Factor 

94.36 

34 

.000 

.931 

.071 

SBCVT 

50.44 

29 

.008 

.957 

.041 

63B  Light-Wheel 

One  Factor 

35.19 

27 

.134 

.979 

.034 

624 

Vehicle  Mechanic 

Two  Factor 

34.87 

26 

.114 

.978 

.034 

SBVT 

25.55 

23 

.323 

.982 

.028 

71L  Administrative 

One  Factor 

47.04 

14 

.000 

.958 

.047 

641 

Special ist 

Two  Factor 

25.76 

13 

.018 

.975 

.033 

SBT 

24.79 

11 

.010 

.972 

.033 

88M  Motor  Transport 

One  Factor 

20.17 

5 

.001 

.959 

.040 

588 

Operator*^ 

SBV 

12.93 

4 

.012 

.967 

.030 

91A  Medical  Specialist‘s 

SBT 

794 

95B  Mi  1  i tary  Police 

One  Factor 

7.10 

9 

.627 

.988 

.023 

444 

Two  Factor 

7.01 

8 

.536 

.986 

SBVT 

2.16 

6 

.905 

.994 

«):  . 

*  The  one-factor  model  teots  the  fit  of 

a  single  factor 

for  all  of 

the  variance 

In  Hands-On 

Test  SCOT 

OS. 

The  two-factor  mode)  proposes  a  Basic 

(across  MOS)  factor  and  a 

Technical  (MOS-specif 1c) 

factor.  ' 

lie 

third  model  factors  are  S  -  Safety/Su 

'■vival,  8  -  Basic  Soldiering,  C  • 

Communications,  1 

-  Identify 

1  Targets,  V  •  Vehicles,  and  T  •  Technical. 

MOS  116  has  no  Basic  Task  Construct. 

MOS  88M  has  no  Technical  Task  Factor. 

MOS  9lA  hands-on  data  were  overfit  by  all  three  models 
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As  another  way  of  looking  at  these  results,  Tables  4.25  and  4.26  show 
the  improvement  in  fit  provided  by  each  model  for  the  job  knowledge  and  hands- 
on  data,  respectively. 

Examination  of  the  above  results  from  LV!  argues  for  the  retention  of 
the  six  Task  Factor  scores  for  both  the  Hands-on  and  Job  Knowledge  measures. 
That  is,  the  basic  scores  for  HO  and  JK  that  serve  as  input  for  the  LVI 
performance  modeling  analysis  will  be  the  same  as  they  were  for  the  CVI 
performance  model  analysis.  Overall,  based  on  Tables  4.23  -  4.26,  not  much 
information  is  lost  when  the  Functional  Categories  are  aggregated  into  the  six 
Task  Factor  scores,  but  the  degree  of  information  loss  increases  when  the 
number  of  scores  is  reduced  from  six  to  two.  Consequently,  the  six-score 
solution  was  retained.  It  preserves  the  most  information  with  fewest  scores 
and  it  is  consistent  with  how  the  HO  and  JK  performance  measures  were  scored 
in  CVI. 


FINAL  ARRAY  OF  LVI  BASIC  PERFORMANCE  SCORES 

This  chapter  has  described  the  development  of  the  basic  scores  for  each 
of  the  first-tour  performance  criterion  measures  that  were  used  to  assess  the 
LVI  sample.  These  are  the  scores  that  were  put  through  the  final  editing  and 
score  imputation  procedures  for  the  LVI  data  file.  They  are  also  the  scores 
that  form  the  basis  for  the  confirmatory  tests  of  the  LVI  model  of  first-tour 
job  performance.  A  summary  list  of  the  basic  performance  scores  produced  by 
the  analyses  described  in  this  chapter  is  given  in  Table  4.27. 

The  lotcnt  structure  model  that  best  fits  the  data  (i.e.,  the  pattern  of 
covariation  among  the  basic  scores)  will  dictate  how  composites  of  these 
scores  should  be  computed.  The  composites,  which  serve  as  the  best  estimates 
of  the  latent  variables,  are  the  criterion  scores  against  which  the 
Experimental  Predictor  Battery  will  be  validated.  The  model  building  and 
validation  results  are  described  in  Chapters  6  and  7  respectively. 


Table  4.25 


Comparison  of  Goodness-of-Fit  for  Three  Models  for  LVI  Job  Knowledge  Tests  for 
10  HOS  _ 


MOS 

Models  Compared* 

Chi-Square 

df 

P 

IIB 

Infantryman^ 

One 

Factor 

vs.  SBCI 

13.03 

2 

.001 

13B 

Cannon  Crewman 

One 

Factor 

vs.  Two  Factor 

40.07 

1 

.000 

One 

Factor 

vs.  SBCIT 

56.01 

6 

.000 

Two 

Factor 

vs.  SBCIT 

15.94 

5 

.007 

19E 

M60  Armor  Crewman 

One 

Factor 

vs.  Two  Factor 

4.79 

1 

.029 

One 

Factor 

vs.  SBCIT 

18  91 

6 

.004 

Two 

Factor 

vs.  SBCIT 

14.12 

5 

.015 

19K 

Ml  Armor  Crewman 

One 

Factor 

vs.  Two  Factor 

5.32 

1 

.021 

One 

Factor 

vs.  SBCir 

13.15 

6 

.041 

Two 

Factor 

vs.  SBCIT 

7.83 

5 

.166 

31C 

Single  Channel 

One 

Factor 

vs.  Two  Factor 

21.42 

1 

.000 

Radio  Operator 

One 

Factor 

vs.  SBCVIT 

48.51 

9 

.000 

Two 

Factor 

vs.  SBCIVT 

27.09 

8 

.001 

63B 

Light-Wheel 

One 

Factor 

vs.  Two  Factor 

4.85 

1 

.028 

Vehicle  Mechanic 

One 

Factor 

vs.  SBVT 

23.31 

4 

.000 

Two 

Factor 

vs.  SBVT 

18.46 

3 

.000 

71L 

Administrative 

One 

Factor 

vs.  Two  Factor 

29.73 

1 

.000 

Special ist 

One 

Factor 

vs.  SBT 

31.88 

3 

.000 

Two 

Factor 

vs.  SBT 

2.15 

2 

.341 

88M 

Motor  Transport' 

One 

Factor 

vs.  SBIV 

3.87 

2 

.144 

91A 

Medical  Specialist 

One 

Factor 

vs.  Two  Factor 

3.77 

1 

.052 

One 

Factor 

vs.  SBIVT 

25.95 

6 

.000 

Two 

Factor 

vs.  SBIVT 

22.18 

5 

.000 

95B 

Mi  1 itary  Police 

One 

Factor 

vs.  Two  Factor 

7.20 

1 

.007 

One 

Factor 

vs.  SBCIVT 

32.33 

9 

.000 

Two 

Factor 

vs.  SBCIVT 

25.13 

8 

.001 

*  The  one-factor  model  tests  the  fit  of  a  single  factor  for  all  of  the  variance  in  Job  Knowledge  Test 
scores.  The  two-factor  model  proposes  a  Basic  (across  MOS)  factor  and  a  Technical  (MOS-specif ic)  factor. 
The  third  model  factors  are  S  •  Safety/Survival.  8  •  Basic  Soldiering.  C  -  Conninicatirns,  I  -  Identify 
Targets.  V  •  Vehicles,  and  T  -  Technical. 

MOS  IIB  has  no  Basic  Task  Construct. 

M9S  88M  has  no  Technical  Task  Construct. 
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Table  4.26 


Comparison  of  Goodness-of-Fit  for  Three  Models  for  LVI  Hands-On  Tests  for  10 
HOS 


MOS 

Models  Compared* 

Chi-Square  df 

_  P 

IIB 

Infantryman^ 

One 

Factor 

vs.  SBC 

3.86 

1 

.049 

13B 

Cannon  Crewman 

One 

Factor 

vs.  Two  Factor 

10.79 

1 

.001 

One 

Factor 

vs.  SBCT 

27.26 

4 

.000 

Two 

Factor 

vs.  SBCT 

16.47 

3 

.000 

19E 

M60  Armor  Crewman 

One 

Factor 

vs.  Two  Factor 

0.00 

1 

.999 

One 

Factor 

vs.  SBCT 

6.80 

4 

.147 

Two 

Factor 

vs.  SBCT 

6.80 

3 

.079 

19K 

Ml  Armor  Crewman 

One 

Factor 

vs.  Two  Factor 

0.00 

1 

.999 

One 

Factor 

vs.  SBCT 

8.30 

4 

.081 

Two 

F  actor 

vs.  SBCT 

8.30 

3 

,040 

31C 

Single  Channel 

One 

Factor 

vs.  Two  Factor 

15.69 

1 

.000 

Radio  Operator 

One 

Factor 

vs.  SBCVT 

59.61 

6 

.000 

Two 

Factor 

vs.  SBCVT 

43.92 

5 

.000 

63B 

L ight-Whee 1 

One 

Factor 

vs.  Two  Factor 

0.32 

1 

572 

Vehicle  Mechanic 

One 

Factor 

vs.  SBVT 

9.64 

4 

.047 

Two 

Factor 

vs.  SBVT 

9.32 

3 

.025 

71L 

Administrative 

One 

Factor 

vs.  Two  Factor 

21.28 

1 

.000 

Specialist 

One 

Factor 

vs.  SBT 

22.25 

3 

.000 

Two 

Factor 

vs.  SBT 

0.97 

2 

.616 

88M 

Motor  Transport‘d 

One 

Factor 

vs.  SBV 

7.24 

1 

.007 

91A 

Medical  Specialist‘s 

95B 

Mi  1 i tary  Police 

One 

Factor 

vs.  Two  Factor 

0.09 

1 

.764 

One 

Factor 

vs.  SBvr 

4.94 

3 

.176 

Two 

Factor 

vs.  SBVT 

4.85 

2 

.088 

Ti,  '  ^♦or  model  tests  the  fit  of  a  single  factor  for  all  of  the  variance  in  Hands-On  Test  scores. 
Tilt  -or  model  proposes  a  Basic  (across  MOS)  factor  and  a  Technical  (MOS-specif  ic)  factor.  The 

third  inodel  factors  are  S  •  Safety/Survival,  0  -  Basic  Soldiering,  C  •  Coiimr  i  cat  ions .  1  •  Identify 
Targets,  V  -  Vehicles,  and  T  -  Technical. 

MOS  118  has  no  Basic  Task  Construct. 

MOS  88M  has  no  Technical  Task  Construct. 

°  LlSREL  overfit  all  three  models  for  91A  hands-on  component  data. 
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Table  4.27 


Basic  Criterion  Scores  Obtained  by  Analyzing  the  Psychometric  Characteristics 
of  the  LVI  Performance  Measures  _ 


ARCHIVAL  DATA  OBTAINED  FROM  THE  PERSONNEL  FILE  FORM 

Number  of  awards  and  certificates 
Physical  Readiness  Test  score 
Number  of  Articles  15  and  Flag  Actions 

Individual  promotion  rate  (i.e.,  deviation  frn  overall  mean) 

Mi6/M19  Qualification  score 

SCORES  OBTAINED  FROM  RATINGS  MEASURES 

Effort  and  Leadership  (ELS)  Factor  from  Army-Wide  BARS 

Maintaining  Personal  Discipline  (MPO)  Factor  from  Army-Hide  BARS 

Physical  Fitness  and  Military  Bearing  (PFB)  Factor  from  Army-Wide  BARS 

Average  of  MOS-Specific  BARS  scales 

Overall  Effectiveness  Rating 

NCO  Potential  Rating 

Combat  Performance  Prediction  Rating  (males  only) 

SCORES  OBTAINED  FROM  HANDS-ON  MEASURES 

Core  Technical  (MOS-specif ic)  score 
Communications  score 
Vehicle  operation  and  maintenance  score 
Basic  skills  (general  soldiering)  score 
Safety  and  survival  score 

SCORES  OBTAINED  FROM  JOB  KNOWLEDGE  MEASURES 

Core  Technical  (MOS-specif ic)  score 

Communi rat  ions  score 

Vehicle  coerarion  and  maintenance  score 

Basic  skill.-,  (general  soldiering)  score 

Identifying  target  and  threat  vehicles  and  aircraft 

Safety  and  survival  score 
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Chapter  5 

FINAL  DATA  EDITING  AND  SCORE  IMPUTATION 

Diane  Steele  and  Randolph  K.  Park 


This  chapter  describes  the  final  data  editing  and  score  imputation 
procedures  for  the  Longitudinal  Validation  first-tour  (LVI)  job  performance 
measures  and  the  Experimental  Battery  (EB)  of  new  predictor  measures 
administered  to  the  LVI  sample  at  the  time  of  entry.  The  performance  data 
will  be  discussed  first,  followed  by  a  review  and  summary  of  what  was  done 
with  the  Experimental  Battery  predictory  data,  which  was  discussed  at  some 
length  in  a  previous  report  (Campbell  &  Zook,  1990).  The  third  and  final 
section  of  this  chapter  presents  a  more  detailed  description  of  the  specific 
imputation  algorithm  that  was  used,  as  well  as  a  before-and-af ter  evaluation 
of  the  effects  of  imputation. 


LVI  PERFORMANCE  MEASURES 

The  Longitudinal  Validation  First-Tour  (LVI)  data  were  collected  from 
11,266  soldiers  in  21  Military  Occupational  Specialties.  For  the  10  MOS 
designated  as  Batch  A  MOS,  the  performance  measures  included: 


-  Hands-On  Tests 

-  Job  Knowledge  Tests 

-  Army-Wide  Ratings 

-  MOS-Specific  Ratings 

-  Combat  Performance  Prediction  Ratings  (males  only) 

-  Job  History 

-  Personnel  File  Form 

-  Army  Job  Satisfaction  Questionnaire 

-  Physical  Requirements 

For  the  11  MOS  designated  as  Batch  Z,  the  performance  measures  included: 

-  School  Knowledge  Tests 

-  Army-Wide  Ratings 

-  Combat  Performance  Prediction  Ratings  (males  only) 

-  Personnel  File  Form 

-  Army  Job  Satisfaction  Questionnaire 

-  Physical  Requirements 

The  construction  of  each  of  these  measures  has  been  described  in  detail 
in  previous  reports  (Campbell,  1987;  Campbell  &  Zook,  1990).  Briefly,  the 
performance  measures  can  be  described  as  follows: 


Hands-On  Tests.  Observation  and  scoring  of  performance  on  13-16 
careful ly  sampled  job  tasks  per  MOS.  Each  MOS  was  tested  on  a 
combination  of  MOS-specific  and  general  soldiering  job  tasks. 

Job  Knowledge  Tests.  Written  tests  of  facts  and  procedures  on  24-30 
carefully  sampled  job  tasks  per  MOS.  Each  MOS  was  tested  on  a 
combination  of  MOS-specific  and  general  soldiering  facts  and 
procedures. 
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School  Knowledge  Tests.  Written  tests  of  115-180  facts  and 
procedures  for  job  tasks  taught  during  training  for  the  MOS. 

Army-Wide  Ratings.  Ratings  of  performance  by  peers  and/or 
supervisors  on  Army-wide  dimensions,  and  the  ratee's  potential 
effectiveness  as  an  NCO. 

MOS-Specific  Ratings.  Ratings  of  performance  by  peers  and/or 
supervisors  on  7-13  MOS-specific  dimensions. 

Combat  Ratings.  Ratings  of  predicted  performance  in  combat 
situations  by  peers  and/or  supervisors  on  19  dimensions.  These  were 
collected  for  males  only. 

Job  History.  Self-report  of  the  frequency  and  recency  of 
performance  for  26-30  MOS-specific  job  tasks. 

Personnel  File  Form.  Self-report  of  administrati ve  and  personnel 
information  including  letters  and  commendations,  Physical  Readiness 
Test  score,  Marksmanship  score,  and  disciplinary  actions. 

Job  Satisfaction.  Self-report  of  satisfaction  with  dimensions 
relevant  to  the  Army,  and  informa  ion  about  soldier's  background. 

Physical  Reou irements .  Self-report  and  supervisor  report  of  the 
examinee's  ability  to  perform  tasks  required  of  the  MOS. 

The  performance  measure  data  are  also  known  as  the  criterion  data.  Both 
of  these  terms  are  used  to  differentiate  these  data,  which  measure  actual 
performance,  from  the  predictor  data,  which  are  used  to  predict  performance. 

Extent  of  Missing  Data 

There  were  6,815  Batch  A  examinees  and  4,451  Batch  Z  examinees. 

Extensive  efforts  were  made  to  collect  complete  information  from  each  examinee 
for  all  instruments.  However,  as  with  all  data  collection  exercises, 
circumstances  precluded  complete  success.  For  any  individual  instrument, 
information  could  be  partially  or  completely  missing.  Some  of  the  chief 
reasons  for  incomplete  information  are  listed  in  Figure  5.1. 

The  final  counts  of  soldiers  for  whom  data  were  analyzed  for  each 

instrument  are  given  in  Tables  5.1  and  5.2  for  Batch  A  and  Batch  Z  MOS, 

respectively.  The  column  headed  Total  gives  the  total  number  of  soldiers 

tested  in  each  MOS,  on  any  or  all  instruments.  These  soldiers  were  recorded 
in  the  "link"  file,  which  contains  demographic  information  and  information 
collected  by  the  Army,  such  as  ASVAB  and  AFQT  scores,  for  all  soldiers  in  the 
sample.  These  data  could  be  linked  to  the  specific  performance  measure  data 
sots,  which  were  stored  individually.  The  columns  for  each  specific 
instrument  contain  the  final  counts  of  examinees  left  in  each  data  set  after 
the  missing  data  had  been  reconstructed  and  examinees  removed  because  their 
level  of  missing  data  for  that  instrument  was  higher  than  acceptable.  The 
final  column  of  Table  5.1,  headed  Combined  Criteria,  gives  the  count  of  Batch 
A  soldiers  with  Hands-On  or  Job  Knowledge  or  MOS  Ratings  or  Army-Wide  Ratings 
or  Personnel  File  Form  data. 
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Hands-On 


Anticipated  Variation  in  Equipment 

Unanticipated  Variation  in  Equipment 

Soldiers  Not  Available  for  Part  or  All  of  Scheduled  Time 

Equipment  Breakdown  or  Non-availability 

Conditions  Preventing  Testing  of  Some  Soldiers  on  Some  Tests 

Scorer  or  Scoresheet  Error 

Ratings  Data 

-  No  Suitable  Raters  Available 

-  Soldier  Does  Not  Perform  Some  Kinds  of  Tasks 

-  Rater  Not  Following  Instructions 

-  Combat  Rating  Not  Collected  for  Females 

Job/School  Knowledge  Test 

-  Soldiers  Not  Available  for  Part  or  All  of  Scheduled  Time 

-  Soldiers  Exceptionally  Slow  in  Taking  Test 

-  Soldiers  Not  Following  Instructions 


Figure  5.1.  Reasons  why  data  were  incomplete. 


From  these  tables,  the  number  of  soldiers  for  whom  data  on  a  particular 
instrument  were  totally  missing  can  be  determined--that  is,  soldiers  not 
available  for  a  particular  testing  session.  For  example,  only  890  of  the  909 
MOS  118  examinees  participated  in  Hands-On  testing. 

Treatment  of  Hissing  Data 

Within  a  particular  instrument,  data  can  be  missing  for  individual 
measures  in  the  tests.  As  was  shown  in  Figure  5.1,  there  were  various  reasons 
for  these  missing  data.  For  example,  in  the  Hands-On  testing,  pieces  of 
equipment  that  were  needed  to  perform  the  task  could  be  missing  or  non¬ 
functioning,  which  precluded  scoring  for  one  or  more  steps  of  the  task.  In 
the  written  tests,  soldiers  might  skip  over  questions  they  could  not  answer  or 
might  not  complete  the  test  within  the  allotted  time.  Raters  may  have  felt 
unable  to  comment  on  unobserved  performance.  When  completing  self-reports, 
soldiers  might  leave  questions  unanswered  if  they  did  not  know  or  did  not  want 
to  provide  the  requested  information. 

Within  instruments,  various  methods  were  used  to  deal  with  partially 
missing  data.  For  some  instruments,  the  missing  data  were  left  as  missing  and 
no  efforts  were  made  to  reconstruct  the  data.  Instruments  for  which  missing 
data  for  examinee  responses  were  not  reconstructed  include  Personnel  File 
Form,  Job  History,  Job  Satisfaction,  and  Physical  Requireme''ts.  For  the  other 
instruments,  data  were  reconstructed  one  instrument  at  a  time,  sometimes  in 
multiple  stages.  Following  is  an  instrument  by  instrument  description  of  the 
treatment  of  missing  data. 
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LVI  Sawple  Sizes  for  Perfonwance  Measures  for  Batch  A  KOS 
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One  soldier  who  appears  in  the  19K  Job  Knowledge  data  is  designated  as  51 B  in  the  link  File. 
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Rules  and  Procedures  for  Eliminating  Cases 


Hands-On  Measures 


Hands-on  testing  consisted  of  observing  performance  on  MOS-specific  or 
general  soldiering  tasks.  Tasks  were  made  up  of  varying  numbers  of  steps. 
Within  particular  tasks,  varying  amounts  of  data  were  missing  at  the  step 
level.  Tables  detailing  the  amount  of  missing  data  for  each  task  in  each  MOS 
are  recorded  in  Appendix  B. 

For  the  most  part,  there  was  little  missing  data--less  than  5  percent 
of  the  records  for  most  steps.  Where  there  were  significant  amounts  of 
missing  data,  the  reason  was  generally  missing  or  damaged  equipment.  For 
example,  in  the  task.  Put  On/Wear  MOPP  Gear,  soldiers  often  did  not  have  one 
or  all  of  the  pieces  required. 

At  the  step  level,  missing  data  were  imputed  using  a  procedure 
originally  developed  for  the  National  Center  for  Education  Statistics  (now  the 
Center  for  Education  Statistics).  The  procedure  was  developed  using  the 
Statistical  Analysis  System  (SAS)  software  and  was  called  PROC  IMPUTE  (Wise  & 
McLaughlin,  1980).  The  details  of  how  the  imputation  procedure  works  are 
described  in  the  concluding  section  of  this  chapter. 

Within  certain  MOS,  some  tasks  were  treated  separately.  For  the  MOS 
118  task.  Day  and  Night  Surveillance,  and  the  MOS  71L  task.  Typing  Speed,  data 
were  not  imputed  at  the  step  level.  These  tasks  did  not  have  r.tep- level  data 
like  the  other  tasks,  but  rather  were  calculated  as  total  scores  at  the  step 
level,  that  is,  total  number  of  targets  located,  and  total  number  of  words  per 
minute. 

The  task.  Put  On/Wear  MOPP  Gear,  was  a  problem  at  many  locations.  For 
this  task,  soldiers  often  did  not  have  all  of  the  equipment  needed  to  perform 
the  entire  task.  The  individual  equipment  items  included  trousers,  jacket, 
boots,  gloves,  and  mask.  Because  individual  pieces  could  be  missing,  some 
scorers  scored  the  performance  measures  for  that  piece  by  having  the  examinee 
pantomime  or  describe  the  correct  procedure.  Others  marked  the  score  sheet 
"Not  Applicable"  and  scored  only  those  sections  where  the  examinee  actually 
had  the  gear.  Rather  than  penalize  the  soldier  for  the  entire  task  because  of 
a  missing  piece  of  equipment,  soldiers  were  given  credit  for  completing  the 
task  and  missing  data  were  imputed,  if  they  actually  performed  two  of  the 
three  parts  of  the  task. 

The  task,  Set  Headspace  and  Timing  on  an  M2  Caliber. 50  Machinegun,  was 
tested  only  on  mechanized  infantry  for  MOS  IIB.  Our  data  did  not  include  an 
indicator  of  mechanized  vs.  non-mechanized  infantry.  Therefore,  if  this  task 
was  administered  at  a  particular  site,  we  assumed  everyone  at  that  site  was 
required  to  perform  this  test.  Where  the  task  was  not  administered,  we  did 
not  attempt  to  impute  data. 

Using  the  imputation  procedure,  data  were  reconstructed  for  all 
examinees  with  missing  data.  Using  this  complete  step-level  data,  task  scores 
were  calculated  as  the  percentage  of  steps  the  examinee  performed  correctly. 
However,  if  an  examinee's  task  score  was  based  on  more  than  approximately  15 
percent  imputed  data,  the  task  score  was  set  to  missing.  The  amount  of 
missing  step-level  data  allowed  per  task  varied,  and  ranged  from  15  to  21 
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percent.  For  the  most  part,  however,  to  receive  a  task  score  at  this  point  in 
the  process,  examinees'  scores  had  to  be  based  on  85  percent  "real "--that  is, 
non- imputed--data  at  the  step  level. 

The  amount  of  missing  task-level  data  after  imputing  the  step  level  and 
calculating  task  scores  is  shown  in  Table  5.3.  Some  MOS  have  tracked  tasks, 
for  example,  13B,  in  which  a  soldier  can  work  with  one  of  three  howitzers, 
M109,  M110A2,  or  M198.  Differences  in  howitzer  model  necessitate  differences 
in  the  specific  tasks.  So  while  all  examinees  perform  16  tasks,  five  of  the 
tasks  are  designed  to  be  specific  to  the  howitzer  type.  In  Table  5.3,  the 
amount  of  task-  level  missing  data  is  separated  by  those  tasks  which  everyone 
in  an  MOS  performs  in  the  same  fashion.  Common  Tasks,  and  those  which  are 
individual izea  because  of  equipment  differences.  Tracked  Tasks.  For  MOS  13B 
data,  there  are  three  columns  to  differentiate  the  tracked  tasks,  but  the 
amount  of  missing  data  for  those  tasks  that  are  not  tracked  is  given  only 
once. 

The  task  number  as  listed  in  the  table  has  no  meaning  except  to  mark 
the  row.  It  has  no  significance  within  any  MOS.  Row  1  is  merely  the  first 
task  that  is  listed  for  any  MOS,  and  that  first  task  may  be  different  for  each 
MOS,  or  for  successive  samples. 

Task  scores  were  imputed  using  the  same  imputation  procedure  that  was 
used  for  steps  within  tasks.  At  this  point,  examinees  were  ve'eted  from  the 
data  set  if  they  had  too  many  tasks  with  scores  derived  from  imputed  data.  By 
MOS,  soldiers  were  removed  from  the  data  set  if  they  had  three  to  five  tasks 
based  on  more  than  15  percent  imputed  step-level  data.  Therefore,  cluster 
scores  and  CVBITS  scores  were  calculated  only  for  soldiers  who  had  85  percent 
of  their  task  scores  based  on  approximately  85  percent  real  data. 

Table  5.4  shows  comparable  missing  data  from  the  Concurrent  Validation 
First-Tour  (CVI)  data.  Again,  note  that  the  task  number  on  Tables  5.3  and  5.4 
is  used  only  to  mark  the  row.  Between  the  two  tables,  the  tasks  may  not  be  in 
the  same  order  within  MOS.  In  addition,  there  were  changes  in  the  tasks 
between  the  two  data  collections.  Some  tasks  were  dropped,  some  tasks  were 
added,  and  some  tasks  were  updated  to  reflect  the  most  current  training 
methods.  Despite  these  differences  in  detail,  the  patterns  of  missing  data  at 
the  task  level  can  be  compared  in  the  two  tables. 

This  comparison  is  presented  in  Table  5.5,  which  shows  that  the 
patterns  of  missing  data  at  the  task  level  were  fairly  similar  between  the  CVI 
data  collection  and  the  LVI  data  collection.  Overall,  there  appears  to  be 
somewhat  more  missing  data  for  the  LVI  sample  than  for  the  CVI  sample.  The 
average  percentage  missing  per  task  for  LVI  MOS  ranges  from  1.62  to  13.91,  and 
for  CVI  MOS  ranges  from  0.05  to  10.71.  The  means  these  MOS  averages  are 
7.03  percent  for  LVI  and  3.83  percent  for  CVI.  With’n  some  MOS,  there  are 
specific  tasks  with  large  amounts  of  missing  data,  generally  because  of 
equipment  failures,  but  most  tasks  are  missing  less  than  5  percent  of  the  data 
(76  of  139  tasks  have  less  than  5  percent  missing  data). 


LVI  Hands-On  Data:  Amount  of  Missing  Data  at  the  Task  Level  After  Step-Level  Imputes  Have  Been  Made,  by  HOS 
(percent)  _ 
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Note:  The  task  number  does  not  designate  the  same  task  for  each  MOi).  o;.«.  ,s  u^ed  n‘V  to  id''ntify  the  row. 
The  task,  Set  Headspace  and  Timing  on  an  M2  Caliber  .50  Machinegu'*.  ijot  ir.puttd  at  the  task  level. 
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Table  5^5 


Comparison  of  Hissing  Hands-On  Task-Level  Data  in  the  LVI  and  CVI  Data 
Collections  _ 


Percent 

of  Data  Missing 

(Mean/Standard  Deviation) 

MOS 

LVI 

CVI 

IIB 

3.52/  4.96 

2.43/  3.84 

13B  (common) 

4.04/  3.64 

2.99/  2.36 

13B  (tracked) 

9.58/  8.17 

3.37/  3.20* 

1.62/  2.87 

1.69/  1.60 

19K^ 

5.09/  7.44 

31C 

13.12/  8.81 

4.85/  5.61 

63B 

13.91/12.68 

5.95/  2.76 

71L 

7.02/  6.33 

2.73/  2.42 

9.86/  7.89 

10.71/12.15 

91A 

2.20/  3.09 

0.05/  0.13 

95B 

5.45/  9.79 

3.56/  4.53 

Overal  1 

7.03/ 

3.83/  2.76 

^  Excludes  track  M102,  which  was  not  collected  in  LVI. 
^  Not  collected  during  CVI. 

^  Collected  as  MOS  64C  in  CVI. 

^  Excludes  19K. 


Job  Knowledoe/School  Knowledge 

The  job  knowledge  and  school  knowledge  test  data  were  processed  using 
similar  methods.  In  these  tests,  there  were  two  reasons  for  partial  missing 
data.  First,  an  examinee  might  skip  over  individual  questions  within  a  test, 
and  second,  an  examinee  might  not  complete  the  test.  The  procedure  used  was 
to  determine  the  total  amount  of  missing  data,  and  then  to  reconstruct  data 
only  for  those  missing  less  than  10  percent  of  the  data.  Tables  5.6  and  5.7 
show  by  MOS  the  counts  of  soldiers  with  missing  data  for  job  knowledge  and 
school  knowledge  tests,  respectively. 

In  addition  to  determining  how  much  missing  data  an  examinee  had  for  the 
test,  we  also  determined  those  who  appeared  to  be  randomly  responding  to  the 
test.  A  random  response  index  was  defined  as  the  correlation  between  the  item 
score  (1  for  correct  and  0  for  incorrect)  and  item  difficulty  expressed  as  the 
proportion  of  subjects  who  answered  the  item  correctly.  For  most  examinees, 
the  correlation  was  positive  since  there  was  a  tendency  to  get  the  easier 
items  correct  and  miss  the  more  difficult  items.  In  a  few  instances  this 
correlation  was  essentially  zero,  suggesting  random  responding.  The  records 
for  those  individuals  were  flagged.  He  did  not,  however,  delete  any  examinee 
records  for  suspected  random  responding.  Only  those  who  were  missing  more 
than  10  percent  of  the  items  had  their  scores  deleted  from  the  jr b  knowledge 
data  or  the  school  knowledge  data. 


no 


Table  5.6 


Number  of  LVI  Cases  With  Incomplete  Job  Knowledge  Data  for  Batch  A  HOS 


MOS 

None 

Missing 

Less  Than 

10%  Missing 

More  Than 

10%  Missing 

IIB 

677 

218 

12 

138" 

614 

280 

11 

19E 

198 

50 

1 

19K 

619 

194 

11 

31C 

377 

127 

24 

638 

523 

200 

29 

71L 

243 

421 

14 

88M 

541 

133 

7 

91A 

595 

203 

26 

95B 

332 

114 

6 

^  13B  includes  84  examinees  for  howitzer  M102  who  were  dropped  from  the  final 
analysis  data. 


Table  5.7 


Number  of  LVI  Cases  With  Incomplete  School  Knowledge  Data  for  Batch  2  HOS 


MOS 

None 

Missing 

Less  Than 

10%  Missing 

Mora  Than 

10%  Missing 

128 

828 

7 

6 

16S 

460 

9 

3 

27E 

85 

3 

2 

29E 

94 

5 

13 

518 

206 

1 

6 

548 

430 

17 

2 

558 

273 

3 

3 

67N 

190 

7 

0 

76Y 

760 

17 

11 

948 

819 

12 

1 

968 

122 

1 

5 

In  most  cases,  examinees  finished  the  job  knowlecne  test.  Th  ie  with 
missing  data  were  individuals  who  had  skipped  over  sections  of  the  test. 
Because  the  amount  of  not-reached  items  was  so  small,  it  was  decided  to  leave 
any  of  those  missing  items  as  missing.  Items  which  were  skipped  within  the 
test  were  reconstructed  by  assigning  a  value  equal  to  the  chance  score.  For 
example,  if  a  question  had  five  response  choices,  the  examinee  had  a  .2  chance 
of  responding  correctly. 


Ill 


The  imputation  procedure  was  used  for  only  three  tasks  within  the  job 


knowledge  tests, 
sites.  Figure  5.2 

The  questions  for  these  tasks  were  not  administered 
gives  the  tasks  for  which  data  were  imputed. 

at  al  1 

MOS 

TASK 

Description 

IIB 

XKG3 

Identify  Friendly/Threat  Vehicles.  This  task  was 
administered  at  only  half  of  the  sites.  The  data 
imputed  for  all  other  sites. 

only 

were 

63B 

HKI2 

Expedient  Repairs.  This  task  was  not  administered 
Bragg.  The  data  for  that  site  were  imputed. 

at  Fort 

71L 

AKHI 

File  Documents  with  MARKS.  Items  4-6  of  this  task 
not  administered  at  all  sites.  The  data  for  those 
missing  the  items  were  imputed. 

were 

sites 

Figure  5.2. 

Tasks 

with  imputed  data  in  the  Job  Knowledge  tests. 

Ratings  Data 

The  Army-wide  ratings  and  the  MOS-specific  ratings  were  the  only  ones 
used  in  the  combined  criteria  files.  Combat  ratings  were  not  included  because 
they  were  not  administered  to  all  examinees;  only  males  were  rated  on  the 
combat  dimensions. 

Katings  were  processed  identically  for  both  Army-wide  and  MOS  ratings. 
For  each  examinee,  the  ratings  received  on  the  individual  dimensions  were 
averaged  for  peer  ratings  separately  from  supervisor  ratings.  To  provide 
combined  scores,  mean  differences  between  peer  and  supervisor  ratings  on  each 
dimension  were  eliminated  by  subtracting  the  peer  mean  from  the  supervisor 
mean  and  dividing  the  result  by  2;  the  absolute  value  was  then  subtracted  from 
the  higher  mean,  whether  peer  or  supervisor,  and  added  to  the  lower  mean. 

These  corrected  dimension  means  were  then  averaged  to  provide  the  combined 
dimension  means. 

Overall  means  were  calculated  as  the  mean  of  all  dimensions  for  peers, 
supervisors,  and  combined.  In  addition,  for  the  Army-wide  ratings,  three 
factor  scores  were  calculated  as  the  means  of  specific  dimensions.  These 
factor  scores  were  Technical  Skill  and  Effort,  Integrity  and  Self-Control,  and 
Appearance. 

If  an  examinee  was  missing  data  on  more  than  20  percent  of  the 
individual  dimensions  for  peers  or  for  supervisors,  the  overall  scores  for 
that  rater  type  were  set  to  missing,  and  the  combined  scores  were  recalculated 
to  reflect  only  the  remaining  scores. 
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Combined  Criteria 


Data  for  each  individual  performance  measure  were  processed  individually. 
After  processing  was  completed  for  these  individual  measures,  they  were 
combined  so  that  all  performance  data  for  each  examinee  were  included  in  a 
single  file.  The  data  were  combined  separately  by  MOS. 

Data  for  the  Batch  A  MOS  were  combined  in  order  to  calculate  first  the 
basic  scores  and  second  the  overall  criterion  scores.  By  MOS,  the  various 
criterion  files  were  combined  by  including  all  examinees  who  were  in  the  Hands- 
On  or  Job  Knowledge  or  Army-Wide  ratings  or  MOS  ratings  or  Personnel  File  Form 
data.  Presence  in  any  individual  file  defined  presence  in  the  combined  file. 

When  the  data  were  combined,  basic  scores  were  calculated  for  the 
individual  performance  measures.  Basic  scores  are  scores  that  are  de.'ived  from 
each  individual  type  of  data.  Table  5.8  shows  the  basic  scores  that  were 
calculated  for  the  five  types  of  performance  data.  It  also  shows,  by 
instrument  and  MOS,  the  amount  of  missing  data  present  for  the  basic  scores 
after  the  files  were  combined.  For  the  Hands-On  and  Job  Knowledge  CVBITS 
scores,  dash  marks  indicate  where  a  ^articular  score  was  not  calculated  for 
that  MOS.  The  imputation  procedure  was  used  to  construct  the  missing  basic 
scores.  Each  MOS  was  imputed  separately. 

Five  overall  criterion  scores  were  calculated  for  each  examinee.  These 
scores  are  General  Soldiering  Proficiency,  Core  Technical  Proficiency,  Effort 
and  Leadership,  Maintaining  Personal  Discipline,  and  Physical  Fitness  and 
Military  Bearing.  (The  Core  Technical  Proficiency  score  was  not  calculated  for 
examinees  in  MOS  IIB  because  it  correlates  highly  with  General  Soldiering 
Proficiency.)  These  scores  are  derived  from  the  basic  scores  and  represent  the 
highest  level  scores  for  the  performance  data. 

Criterion  scores  were  calculated  only  where  there  was  less  than  50 
percent  imputed  data  at  the  basic  score  level  for  those  scores  included  in  the 
individual  criterion  score.  For  example,  for  MOS  IIB,  the  Effort  and 
Leadership  score  was  calculated  from  four  of  the  basic  scores.  If  an  examinee 
had  imputed  data  for  two  or  more  of  the  basic  scores,  the  Effort  and  Leadership 
score  was  not  calculated. 

Table  5.9  shows  tht  amount  of  missing  data  in  the  final  sets.  These  data 
sets  are  referred  to  as  the  "combined  criteria  data"  because  they  include  all 
of  the  overall  criterion  scores. 


LV  PREDICTOR  DATA 

In  addition  to  the  performance  data,  missing  Longitudinal  Validation 
predictor  data  were  also  imputed.  For  a  complete  description  of  the  editing 
process  used  on  the  predictor  data,  see  the  FY90  Annual  Report.  The  bulk  of 
the  editing  process  was  accomplished  during  FY90,  but  additional  work  was  done 
during  FY91. 

As  described  in  the  FY90  Annual  Report,  scale-level  scores  were  calculated 
using  the  same  methods  as  had  been  used  during  the  CV  analyses.  A  scale-level 
score  is  the  score  or  scores  derived  from  an  individual  test;  for  example,  15 
scale  scores  were  calculated  from  the  ABLE  test.  At  the  time,  due  to  a  change 


113 


LV I  Combined  Criteria  Data:  Amount  of  Missing  Data  for  Basic  Scores,  bv  MOS  (percent 
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Note.  --  indicates  that  the  particular  score  was  not  calculated  for  that  MOS. 


Table  5.9 


LVI  Combined 
(percent) 

Criteria 

Data: 

Amount 

of  Hissing 

Data 

for  Final  Scores, 

by  MOS 

118 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

General 

Soldier 

Proficiency 

.0 

9.5 

.0 

.4 

1.9 

.7 

.6 

.1 

.4 

.0 

Core 

Technical 

Proficiency^ 

9.5 

.0 

.4 

1.9 

.7 

.6 

.1 

.4 

.0 

Effort  and 
Leadership 

1.2 

3.1 

3.2 

5.3 

7.4 

3.5 

7.7 

2.8 

2.4 

.2 

Maintain 

Personal 

Discipline 

.0 

.2 

.0 

.2 

.6 

.4 

.9 

.3 

.1 

.0 

Physical 
Fitness  and 
Bearing 

.0 

.0 

.0 

.1 

.4 

.  3 

.7 

.3 

.0 

.  L, 

^  Core  Technical  Proficiency  was  not  calculated  for  MOS  IIB. 


in  the  software  version  of  SAS  available  on  the  NIH  computer,  the  imputation 
procedure  was  not  available.  We  were  using  a  version  that  was  written  for  the 
PC.  The  practical  implication  of  this  limitation  was  that  vve  could  not  use 
the  entire  predictor  data  set  in  imputing  the  missing  values.  Downloading  the 
data  from  the  mainframe  to  the  PC  would  have  been  too  time  consuming  and 
expensive. 

The  predictor  data  consisted  of  paper-and-penci 1  tests  and  computer- 
administered  tests.  The  paper-and-penci 1  tests  included: 

Spatial  Tests  Non-Spatial  Tests 

Orientation  Test  ABLE 

Map  Test  AVOICE 

Assembling  Objects  Test  JOB 

Reasoning  Test 
Object  Rotation  Test 
Maze  Test 


The  computer-administered  tests  included: 


Simple  Reaction  Time 
Choice  Reaction  Time 
Short-Term  Memory 
Target  Tracking  1 
Target  Tracking  2 


Perceptual  Speed  and  Accuracy 
Number  Memory 
Cannon  Shoot 
Target  Identification 
Target  Shoot 
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The  scale  scores  for  the  paper-and-penci 1  tests  and  the  computer- 
administered  tests  were  imputed  separately,  although  the  same  methodology  was 
used.  The  paper-and-penci 1  scale  scores  were  imputed  first. 


Paper-and-Pencil  Tests 

The  paper-and-penci 1  tests  yielded  65  scale  scores:  15  ABLE  scale 
scores,  23  AVOICE  scale  scores,  7  JOB  scale  scores,  14  alternate  ABLE  factor 
scores^  and  6  spatial  test  scores.  Scale  scores  for  individual  tests  were 
combined  to  create  composite  scores,  which  convert  the  scale  scores  into 
overall  scores  for  each  test.  There  are  seven  ABLE  composites,  eight  AVOICE 
composites,  three  JOB  composites,  and  one  spatial  composite.  Not  all  of  these 
scale  scores  were  used  in  creating  composite  scores,  however.  Table  5.10 
shows  the  amount  of  missing  data  for  those  scale  scores  used  to  create 
composite  scores.  These  scores  were  the  only  ones  used  during  the  imputation 
process  for  the  predictor  data. 

Because  of  the  large  number  of  records  in  the  file  (49,408  observations 
in  the  total  file),  it  was  d-'cided  to  use  a  sample  of  records  to  impute  the 
missing  data  rather  than  the  entire  file.  First,  all  records  in  which  data 
were  missing  for  at  least  one  of  the  scale  scores  were  flagged  to  be  included 
in  the  imputation  sample.  (Records  missing  all  scale  scores  had  no  data 
imputed.)  Second,  a  random  sample  of  half  of  the  observations  in  the  file 
were  also  flagged  for  use  in  imputing  the  missing  data.  To  select  records  for 
the  random  sample,  the  data  were  sorted  by  MOS,  gender,  race,  and  AFQT  score. 
Then,  every  other  record  was  flagged  for  inclusion  in  the  sample.  Because 
some  of  the  records  flagged  as  having  missing  data  could  also  have  been 
flagged  as  part  of  the  random  sample,  the  final  sample  used  for  imputation 
consisted  of  27,112  records.  These  records  were  written  out  as  raw  data  files 
and  downloaded  to  the  PC  for  imputation. 

The  data  were  imputed  battery  by  battery.  That  is,  only  ABLE  data  were 
used  to  impute  ABLE  data,  only  AVOICE  data  were  used  to  impute  AVOICE  data, 
and  so  forth. 

After  the  imputed  scale  score  data  were  added  to  the  files,  the  scale 
scores  were  standardized  to  a  mean  of  50  and  a  standard  deviation  of  10. 
Conoosite  scores  were  then  calculated  using  the  standardized  scale  scores. 
Composite  scores  were  calculated  only  where  50  percent  or  more  of  the  data  in 
the  scale  scores  included  in  the  composite  was  "real"  rather  than  imputed.^ 


^  These  alternate  ABLE  factor  scores  (ABLE-168  and  ABLE-114)  are 
equivalent  to  scale  scores,  but  are  still  being  studied  for  their  inclusion  in 
analyses.  These  scores  were  not  imputed  and  were  not  used  to  calculate 
composite  scores. 

2 

There  are  three  levels  of  "scores"  in  the  paper-and-penci  1  predictor 
data:  (a)  examinee  responses,  (b)  scale  scores,  and  (c)  composite  scores. 
EAaminee  responses  were  not  imputed,  so  scale  scores  that  are  derived  from 
those  responses  could  be  missing.  Missing  scale  scores  were  imputed  in  order 
to  calculate  composite  scores;  however,  if  50  percent  or  more  of  the  scale 
scores  that  made  uo  any  individual  composite  score  were  imputed,  the  composite 
score  was  not  calculated. 
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Table  5.10 


LVI  Predictor  Data;  Amount  of  Hissing  Data  for  Paper-and-Penci 1  Scale  Scores 


Score 

Not  Missing 

Missing 

Assembling  Objects  -  Number  Correct 

49,042 

366 

Map  -  Number  Correct 

49,047 

361 

Maze  -  Number  Correct 

49,052 

356 

Object  Rotation  -  Number  Correct 

49,103 

305 

Orientation  -  Number  Correct 

49,072 

336 

Reasoning  -  Number  Correct 

49,103 

305 

JOB  Scale  1  -  Pride 

46,525 

2,883 

JOB  Scale  2  -  Job  Security/Comfort 

46,634 

2,774 

JOB  Scale  3  -  Serving  Others 

46,295 

3,113 

JOB  Scale  4  -  Job  Autoncry 

46,037 

3,371 

JOB  Scale  5  -  Routine 

45,975 

3,433 

JOB  Scale  6  -  Ambition 

46,058 

3,350 

ABLE  Scale  1  -  Emotional  Stability 

44,264 

5,144 

ABLE  Scale  2  -  Self-Esteem 

44,247 

5,161 

ABLE  Scale  3  -  Cooperativeness 

44,258 

5,150 

ABLE  Scale  4  -  Conscientiousness 

44,199 

5,209 

ABLE  Scale  5  -  Nondelinquency 

44,228 

5,130 

ABLE  Scale  6  -  Traditional  Values 

44,190 

5,218 

ABLE  Scale  7  -  Work  Orientation 

44,260 

5,148 

ABLE  Scale  8  -  Internal  Control 

44,254 

5,154 

ABLE  Scale  9  -  Energy  Level 

44,217 

5,191 

ABLE  Scale  10  -  Dominance 

44,246 

5,162 

ABLE  Scale  11  -  Physical  Condition 

44,264 

5,144 

AVOICE  Scale  1  -  Clerical/Administrative 

45,477 

3,931 

AVOICE  Scale  2  -  Mechanics 

45,941 

1, 157 

AVOICE  Scale  3  -  Heavy  Construction 

45,851 

3  557 

AVOICE  Scale  4  -  Electronics 

45,922 

3,486 

AVOICE  Scale  5  -  Combat 

45,939 

3.469 

AVOICE  Scale  8  -  Medical  Services 

^5,545 

3,863 

AVOICE  Scale  7  -  Rugged  Individualism 

45,944 

3,464 

AVOICE  Scale  8  -  Leadership/Guidance 

45,508 

3,900 

A''OICE  Scale  9  -  Law  Enforcement 

45,958 

3,450 

AVOICE  Scale  10  -  Food  Service  Professional 

45,916 

3,492 

AVOICE  Scale  11  -  Firearms  Enchusiast 

45,942 

3,466 

AVOICE  Scale  12  -  Science/Chemical 

45,970 

3,438 

AVOICE  Scale  13  -  Drafting 

45,976 

3,432 

AVOICE  Scale  14  -  Audiographics 

45,452 

3,956 

AVOICE  Scale  15  -  Aesthetics 

45,279 

4,129 

AVOICE  Scale  IE  -  Computers 

45  554 

3,854 

AVOICE  Scale  17  -  Food  Service  Employee 

45,965 

3,443 

AVOICE  Scale  18  -  Mathematics 

45,691 

3,717 

AVOICE  Scale  19  -  Electronic  Communications 

45,602 

3,806 

AVOICE  Scale  20  -  Warehousing/Shipping 

45,963 

3,445 

AVOICE  Scale  21  -  Fire  Protection 

45,972 

3,436 

AVOICE  Scale  22  -  Vehicle  Operator 

45,971 

3,437 
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Computer-Admin'istered  Tests 

The  computer  battery  yielded  17  scale  scores.  Table  5.11  shows  the 
amount  of  missing  data  for  those  scores. 

Table  5.11 


LVI  Predictor  Data:  Amount  of  Kissing  Data  for  Computer-Adnini stored  Scale 
Scores 


Score 

Not 

Missing 

Missing 

Target  Identification  -  Mea,.  of  Clipped  Decision  Time 

38,401 

513 

Target  Identification  -  Proportion  Correct 

38,404 

510 

Number  Memory  -  Mean  of  Clipped  Operation  Means 

33,324 

590 

Number  Memory  -  Proportion  Correct 

38,353 

561 

Target  Track  1  -  Mean  Log  (Distance-*-! ) 

38,825 

89 

Target  Track  2  -  Mean  Log  (Distance-^1 ) 

38,793 

121 

Cannon  Shoot  -  Mean  Absolute  Time  Discrepancy 

38,603 

311 

Target  Shoot  '  Mean  Log  (Oistance-t-1 ) 

37,477 

1437 

Mean  of  Median  Movement  Times  across  5  tests 

37,863 

1051 

Simple  Reaction  Time  -  Median  Decision  Time 

38,747 

167 

Simple  Reaction  Time  -  Proportion  Correct 

38,747 

167 

Choice  Reaction  Time  -  Median  Decision  Time 

38,856 

58 

Choice  Reaction  Time  -  Proportion  Correct 

38,856 

50 

rerceptua!  Speed/Accuracy  -  Mean  of  Clipped  D'’cision  Time 

38,703 

211 

Perceptual  Speed/Accuracy  -  Proportion  Correct 

38,734 

180 

Short-Term  Memory  -  Mean  of  Clipped  Decision  Time 

38,483 

431 

Short-Term  Memory  -  Proportion  Correct 

38,490 

424 

Because  of  the  large  number  of  records  in  the  file  (38,914  observations 
in  the  total  file),  it  was  decided  to  use  a  sample  of  records  to  impute  the 
missing  dai.a  rather  than  the  entire  file.  First,  all  records  in  which  data 
were  missing  for  at  least  one  of  the  scale  scores  were  flagged  to  be  included 
in  the  imputation  sample.  (Records  missing  all  scale  scores  had  no  data 
imputed.)  Second,  a  random  sample  of  half  of  the  observations  in  the  file 
were  also  flagged  for  use  in  imputing  the  missing  data.  To  select  records  for 
the  random  sample,  the  data  were  sorted  by  MOS,  gender,  race,  and  AFQT  score. 
Then,  every  other  record  was  flagged  for  inclusion  in  the  sample.  Because 
sor  of  the  records  flagged  as  having  missing  data  could  also  have  been 
flagged  as  part  of  the  random  sample,  the  final  sample  used  for  imputation 
consisted  of  20,273  records.  These  records  were  written  out  as  raw  data  files 
and  downlodoed  to  the  PC  toi'  imputation. 
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Unlike  the  paper-and-penci 1  predictor  data  which  were  imputed  separately 
by  battery,  all  computer  scale  scores  were  imputed  together.  As  stated 
earlier,  for  the  paper-and-penci 1  data,  for  example,  only  ABLE  data  were  used 
to  impute  ABLE  data.  In  the  computer  data,  all  scale  score  information  was 
used  to  impute  any  missing  scale  scores. 

After  imputation,  the  scale  scores  were  standardized  to  a  mean  of  50  and 
a  standard  deviation  of  10.  Using  the  standardized  scale  scores,  eight 
composite  scores  were  calculated  where  the  examinee  had  50  percent  of  more 
non- imputed  scale  score  data.^ 


SCORE  IMPUTATION 

In  any  data  analyses,  the  more  data  one  is  able  to  collect,  the  more 
complete  those  analyses  will  be.  Extensive  efforts  had  been  made  during  the 
LVI  data  collection  process  to  obtain  complete  data  across  instruments  and 
within  instruments  for  each  examinee.  While  building  the  analysis  data  sets, 
extensive  efforts  were  made  to  retain  the  data  for  as  many  examinees  as 
possible. 

An  imputation  procedure  was  developed  that  used  existing  data  to 
estimace  values  for  missing  data.  This  procedure,  described  below,  was  used 
in  the  CVI  analyses  to  develop  performance  scores  for  a  very  high  proportion 
of  soldiers  tested  (Wise,  McHenry,  &  Young,  1986).  The  decision  rules  and 
imputation  procedures  used  in  the  CVI  analyses  were  replicated  in  the  LVI 
analyses  as  closely  as  possible.  In  this  way,  it  was  possible  to  develop 
performance  scores  for  a  very  high  proportion  of  the  LVI  soldiers  tested. 

The  Imputation  Procedure 

A  procedure  known  as  PROC  IMPUTE  was  developed  for  the  National  Center 
for  Education  Statistics  (now  the  Center  for  Education  Statistics)  (Wise  & 
McLaughlin,  1980).  PROC  IMPUTE  uses  regression  estimates  to  predict  missing 
values.  Each  missing  value  is  predicted  from  other  values  for  the  subject  in 
question  so  that  individual  differences  are  retained.  The  regression 
coefficient  and  intercept  vary  from  item  to  item  so  that  differences  in  item 
difficulty  are  also  reflected  in  the  predicted  values.  PROC  IMPUTE  adds  a 
random  variable  with  variance  equal  to  the  error  of  estimate  for  predicting 
the  missing  value.  If  such  a  random  variable  is  not  added,  the  imputed  values 
are  more  highly  correlated  with  values  on  other  variables  in  comparison  with 
nonimputed  values. 

PROC  IMPUTE  employs  a  sequential  strategy  that  maintains  relationships 
between  variables  when  more  than  one  value  is  imputed  for  the  same  examinee. 


^There  are  three  levels  of  "scores"  in  the  computer  predictor  data:  (a) 
examinee  responses,  (b)  scale  scores,  and  (c)  composite  scores.  Examinee 
responses  were  not  imputed,  so  scale  scores  that  are  derived  from  those 

responses  could  be  missing.  Missing  scale  scores  were  imputed  in  order  to 

calculate  composite  scores;  however,  if  60  percent  or  more  of  the  scale  scores 

that  made  up  any  individual  composite  score  were  imputed,  the  composite  score 

was  not  calculated. 
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A  two-stage  approach  is  used,  with  the  first  variable  imputed  from  nonmissing 
values.  The  second  (and  subsequent)  variable(s)  is  imputed  from  the 
nonmissing  values  plus  the  imputed  value  for  the  first  variable.  After  all 
initial  imputations,  values  are  reimputed  in  a  second  pass  where  all  of  the 
initially  imputed  values  participate  in  the  reimputation  of  each  missing 
value. 

PROC  IMPUTE  rests  on  having  good  estimates  of  the  correlations  among  the 
variables.  Experience  in  the  imputation  of  missing  values  has  led  to  the 
formulation  of  three  guiding  principles:  (a)  no  more  than  50  variables  at  a 
time  should  be  imputed;  (b)  the  examinee  sample  size  should  be  large  enough  to 
obtain  reasonable  estimates  of  the  correlations;  and  (c)  the  amount  of  missing 
data  for  a  variable  should  not  be  excessive.  In  general,  the  larger  the 
examinee  sample  size  and  the  smaller  the  proportion  of  missing  data,  the 
better  the  estimates  of  the  imputed  values. 

Imputation  was  employed  at  several  levels,  as  was  detailed  earlier  in 
this  chapter.  In  the  Hands-On  data,  missing  steps  scores  were  imputed  so  that 
task  scores  could  be  computed.  Missing  task  scores  were  then  imputed  from 
existing  task  scores.  Missing  Job  Knowledge  test  scores  were  imputed  for 
selected  tests  from  existing  Job  Knowledge  scores.  Missing  criterion  scores 
were  imputed  across  all  instruments;  Hands-On,  Job  KnowK.;  ,  Army-Wide 
Ratings,  MOS  Ratings,  and  Personnel  File  Form.  (For  the  Personnel  File  Form, 
individual  examinee  responses  were  not  imputed;  however,  basic  scores  for  the 
Personnel  File  Form  were  imputed  for  use  in  calculating  overall  criterion 
scores. ) 

In  the  Hands-On  data,  several  issues  arose  that  were  caused  by  the 
nature  of  the  data.  At  the  step  score  level,  some  tasks  had  more  than  50 
variables.  (The  problem  with  the  number  of  variables  is  described  in  more 
oetail  below.)  For  example,  in  MOS  31C,  there  were  73  steps  for  the  task. 
Operate  TTY  GRC-142.  The  approacn  we  chose  was  to  run  PROC  IMPUTE  twice.  The 
first  run  imputed  all  the  odd-numbered  steps  and  the  second  run  imputed  all 
the  even-numbered  steps.  In  MOS  13B,  the  task.  Put  on/Hear  M25  Mask, 
contained  six  steos  and  had  only  16  observations  missing  two  data  points 
Since  this  task  involved  an  alternative  face-mask,  we  choose  to  combine  this 
task  with  the  face-mask  task  performed  by  the  majority  of  examinees  (F'ut 
on/Wear  Ml 7  Mask). 

At  the  task  score  level,  we  ran  -into  an  issue  with  imputing  data  on 
"tracked"  task  scores.  Tracked  tasks  occurred  when  o  soldier  worked  on  one  of 
several  alternative  systems.  For  example,  in  MOS  13B,  a  soldier  might  v/ork  on 
one  of  three  howitzer  types.  While  all  soldiers  performed  common  tasks, 
specific  tasks  were  associated  with  each  howitzer  model.  Soldiers  had  data  on 
their  particular  model  on  wmch  they  were  tested,  but  not  on  the  others. 
Characteristic  to  tracked  tasks  is  the  high  percentage  of  missing  data  when 
dll  the  soldiers  are  considered.  Thus,  imputing  the  scores  on  these  tasks  is 
problematic.  It  also  seemed  illogical  to  impute  scores  on  soldiers  who  were 
not  on  the  trocx  containing  the  task.  The  aoproach  we  took  was  to  use  a  two- 
stage  imputation  process.  First,  the  tasks  common  to  all  examinees  across  all 
“•.racks  were  imouted  to  create  full  data  on  the  commori  tasks.  Then  the  total 
sample  on  the  MOS  was  separated  into  subsamples  of  soldiers  who  represented 
the  different  tracks.  The  second  stage  of  the  imputation  was  run  on  each 
subsample  separately,  using  the  imputed  common  tasks  and  the  specific  tracked 
tasks  to  he  imputed. 
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Imputation  at  the  basic  score  level  (those  scores  listed  in  Table  5.8) 
was  fairly  straightforward.  Sample  sizes  ranged  from  249-907,  the  number  of 
variables  ranged  from  16-21,  and  the  amount  of  missing  data  ranged  from  0-17 
percent.  The  relatively  large  samples  combined  with  the  generally  low 
percentage  of  missing  data  yielded  fairly  good  estimates  of  the  inter¬ 
correlations  among  the  criteria.  Since  the  number  of  variables  was  21  or 
less,  only  one  pass  through  the  imputation  procedure  was  necessary. 

Examining  the  Results  of  Imputation 

The  results  of  the  imputation  were  examined  at  two  levels.  First,  after 
each  PROC  IMPUTE  run,  the  program  output  was  inspected.  Second,  the  pre-imputed 
and  the  post- imputed  data  sets  were  compared.  The  process  for  accomplishing 
this  examination  is  briefly  described  below. 

Inspection  of  Program  Output 

The  PROC  IMPUTE  output  provides  five  reports  for  use  in  evaluating  the 
results:  (1)  missing  data  frequencies  and  univariate  statistics,  (2)  character¬ 
istics  of  cases  with  missing  values,  (3)  correlations  between  reported  values, 
(4)  regression  equations,  and  (5)  conditional  distributions.  A  sample  output 
for  the  imputation  of  step-level  data  for  a  Hands-On  task  in  MOS  71L  is  shown 
as  Appendix  C. 

Missing  data  report  1  is  examined  to  verify  that  the  data  were  read  in 
correctly,  and  that  each  variable  was  not  missing  an  excessive  amount  of  data. 

It  reports  for  each  variable:  percent  missing  data,  frequency  values  of  number 
present,  minimum  and  maximum  values,  and  mean  and  standard  deviation  of  values. 
Report  2  describes  the  relationship  of  each  variable  to  all  other  variables  in 
the  data  set.  From  the  sample  output,  one  of  the  results  can  be  extracted  as 
follows: 


STEP  2 


STEP  1 

59.8183  Mean  of  Step  1  with  cases  missing  Step  2 
11.2480  Standard  deviation  of  Step  1  with  cases  missing  Step  2 
165  Number  of  cases  missing  Step  2  with  values  for  Step  1 

0.232  Phi  correlation  of  missing  data  flags 
2.305  t  significance  of  difference  in  Step  1  between  cases 
with  Step  2 

0.0018  t  significance  of  difference  in  Step  1  between  cases 
with  no  values  for  Step  2 


The  third  report  gives  the  correlations  for  the  pre-imputed  values  of 
all  variables.  This  is  calculated  prior  to  the  imputation  of  the  data. 

Report  4  gives  the  results  for  the  regression  equations  used  in  the  imputation 
of  each  variable.  Regression  functions  are  generated  only  for  variables  with 
some  missing  data.  The  variables  are  ordered  so  as  to  maximize  the  total 
variance  accounted  for  in  the  prediction  of  all  missing  values  when  each 
v.ariflble  is  predicted  only  from  preceding  variables.  This  ordering  is 
necessary  to  ensure  that  any  missing  values  among  the  predictor  values  will 
have  already  been  filled  in  before  the  variable  is  used  as  a  predictor  in  a 
regression  function. 


The  conditional  distribution  report  5,  which  shows  the  cumulative 
di  scr-ibut  ion  of  each  target  value  in  each  regression  value  subset,  is  examined 
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to  verify  the  reasonable  less  of  the  distributions.  Among  the  values  it 
reports  are  the  regressed  data  value,  the  frequency  with  the  data  target,  and 
the  regressed  mean  variable.  Examination  of  report  5  should  show  that  the 
frequencies  are  uniformly  distributed  over  the  range  of  data  values,  and  the 
regressed  mean  variable  should  round  to  the  regressed  value.  The  mean  of  the 
imputed  values  should  also  be  compared  with  the  mean  values  from  report  1. 

The  imputation  results  are  aberrant  when  the  frequency  of  data  values  is 
greater  at  the  extremes  of  the  distribution  (i.e.,  a  U-shaped  frequency 
distribution).  In  this  case,  the  regressed  mean  variables  are  also  nonconver- 
gent  and  become  very  large.  If  the  results  are  aberrant,  then  the  imputation 
procedure  may  have  been  affected  by  the  variables  with  large  percentages  of 
missing  data.  Additional  PROC  IMPUTE  runs  may  be  necessary,  excluding  these 
variables  from  the  imputation. 

This  brief  description  of  the  imputation  inspection  process  provides  the 
information  necessary  for  an  overall  understanding  of  the  procedure.  For  more 
detailed  information,  it  is  suggested  that  the  reader  obtain  copies  of  the 
reports  listed  in  the  references. 

Comparison  of  Data  Sets 

Two  sets  of  pre-  and  post- imputation  comparisons  were  made  for  each  MOS: 
(a)  after  the  hands-on  score  level  imputation,  and  (b)  after  the  criterion 
construct  level  imputation. 

the  means  and  variances  of  the  pre-  and  post-imputation  results  for  the 
hands-on  data  were  compared  for  each  MOS.  An  independent  t  was  used  to 
examine  the  difference  between  means,  and  an  independent  F_  of  the  ratio  of 
variances  was  used  to  examine  differences  between  variances.  As  shown  in 
Table  5.12,  the  _ts  and  £s  are  mostly  nonsignificant  at  the  .05  level.  The 
significance  of  two  £s  and  one  £  may  have  occurred  by  chance  since  the 
algorithm  adds  random  error  to  the  solution.  Differences  between  the  pre¬ 
imputation  correlation  matrix  and  the  post- imputation  matrix  are  reported  in 
Table  5.13.  The  table  reports  the  average  root  mean  square  of  the  difference, 
the  sum  of  the  absolute  differences,  and  the  number  of  correlations 
overestimated  and  the  number  underestimated  by  PROC  IMPUTE.  For  each  MOS,  the 
full  intercorrelation  matrices  of  the  pre-imouted  variables  and  the  residuals 
for  the  pre-  and  post-imputed  results,  and  the  pre- imputed  and  residual  means 
and  variances  are  reported  in  Appendix  0. 

A  similar  analysis  was  done  for  the  differences  between  the  pre-  and 
post-imputation  results  for  each  MOS  at  the  basic  score  level.  Table  5.14 
summarizes  the  results  of  the  jt  and  F  tests  for  differences  between  means  and 
variances,  respectively.  Table  5.15  reports  the  summary  of  the  differences 
between  the  correlation  matrices. 


Table  5.12 


Suouiary  of  Difference  Tests  for  Heans  and  Variances  for  Pre-/Post-Imputation 
Results  on  the  Hands-On  Task  Scores 


Number 

MOS 

of  Tests 

Range  of  t  Values 

Range  of 

£  Values 

IIB 

13 

-.723  -  .223 

-  1.183 

13B 

10 

-.138  -  .133 

-  1.470 

19E 

15 

-.073  -  .092 

1.024 

-  1.291 

19K 

13 

-.661  -  .167 

1.002 

-  1.344 

31C 

14 

-.263  -  .066 

1.061 

-  1.395 

63B 

13 

-.341  -  .223 

1.050 

-  1.235, 

71L 

13 

-.852  -  .094^ 

1.047 

-  1.235*’ 

88M 

15 

-.872  -  .227 

1.025 

-  1.333 

91A 

13 

-.202  -  .189 

1.046 

-  1.193 

958 

12 

-.036  -  .202" 

1.000 

-  1.163 

^  t-test 

value  = 

-1.698 

on  Task  12  and  3.881 

on  1  ask  13. 

^  F-test 

value  = 

2.849  on  Task  13. 

i-test 

value  = 

-1.741 

on  Task  8. 

Table  5. 

13 

Suranary  of  Root  Mean  Square  and  Absolute  Differences  for 

Pre-/Post- Imputation 

Results 

on  the  HandS'On 

Task  Scores 

Absolute 

Differences 

Differences 

MOS 

RMS 

Difference 

>0 

<0 

IIB 

.081 

.065 

51 

27 

13B 

.088 

.074 

25 

20 

19F 

.073 

.056 

47 

56 

19K 

.046 

.036 

35 

43 

31C 

.083 

.062 

46 

45 

63B 

.064 

.050 

35 

43 

71L 

.063 

.052 

53 

25 

88M 

.083 

.063 

53 

52 

91A 

.070 

.056 

62 

16 

95B 

.070 

.052 

32 

34 
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Table  5.14 

Suuinary  of  Difference  Tests  for  Means  and  Variances  for  Pre-/Post-Imputat1on 
Results  on  Basic  Scl 'es  _  _ _ 


Number 


MOS 

of  Tests 

Range  of  t  Values 

Range  of 

Values 

lie 

17 

-.014  -  .142 

1.001 

1.019 

13B 

19 

-.058  -  .181 

- 

1.051 

19E 

19 

1.000 

- 

1.031 

19K 

19 

1.002 

- 

1.027 

31C 

21 

-.134  -  .167 

- 

1.048 

63B 

18 

-.137  -  .081 

1.001 

1.026 

71L 

16 

- 

1.038 

88M 

17 

- 

1.029 

91A 

18 

* 

1.045 

95B 

20 

-.119  -  .075 

- 

1.050 

Table  5.15 

Sutimary  of  Root  Mean  Square  and  Absolute  Differences  for  Pre-/Post-Imputation 
Results  on  Basic  Scores 


MOS 

RMS 

Absolute 

Difference 

Differences 

>0 

Differences 

<0 

IIB 

.007 

.006 

66 

70 

13B 

.013 

.010 

88 

83 

19E 

.021 

.014 

77 

93 

19K 

.012 

.009 

46 

125 

3iC 

.019 

.015 

67 

143 

63B 

.015 

.011 

-1 

102 

71L 

.014 

.011 

19 

81 

88M 

.013 

.009 

60 

76 

91A 

.010 

.008 

68 

85 

95B 

.003 

,006 

82 

108 

Conclusions 

In  terms  of  absolute  size,  the  residuals  summarized  in  Table  5.15  are 
quite  small.  Imputation  makes  virtually  no  difference  in  the  magnitude  of  the 
intercorrelations  among  the  criterion  scores  that  were  used  to  create  the 
performance  factor  scores  in  the  validation  analyses.  At  the  criterion  score 
level,  these  results  are  similar  to  those  obtained  earlier  from  the  CV 
imputation  (Wise,  McHenry,  &  Young,  1986).  The  full  results  for  each  MOS  are 
reported  in  Appendix  E. 
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Chapter  6 

DEVELOPMENT  OF  THE  LONGITUDINAL  VALIDATIOii  SAMPLE 
FIRST-TOUR  PERFORMANCE  MODEL 

Scott  H.  Oppler,  Ruth  A.  Childs,  and  Norman  G.  Peterson 


A  model  of  first-tour  performance  in  entry-level  Army  jobs,  developed 
using  data  from  the  Project  A  Concurrent  Validation  (CVI)  sample,  has  been 
described  by  J.P.  Campbell,  McHenry,  and  Wise  (1990).  This  model  included 
five  performance  factors,  labeled  Cere  Technical  Proficiency  (CTP),  General 
Soldiering  Prof'ciency  (GSP),  Effort  and  Leadership  (ELS),  Maintaining 
Personal  Oiscipl’ne  (MPO),  and  Physical  Fitness  and  Military  Bearing  (PFB). 
Definitions  for  these  factors  are  provided  in  Figure  6.1.  Additionally,  the 
CVI  model  included  two  measurement  method  factors,  a  Ratings  factor  and  a 
Paper-and-Penci 1  Test,  or  Written,  factor.  The  Ratings  factor  was  intended  to 
represent  similarities  among  performance  ratings,  while  the  Written  factor 
represented  similarities  among  performance  scores  based  on  paper-and-penci  1 
measures . 

The  development  of  the  CVI  model  involved  the  following  steps:  (a) 
developing  a  set  of  basic  performance  criterion  scores,  (b)  assigning  these 
basic  variables  to  five  hypothesized  performance  factors  and  two  hypothesized 
method  factors,  and  (c)  refining  the  model  within  and  across  jobs  using 
confirmatory  factor-analytic  techniques.  Because  the  model  was  developed 
using  the  CV  da>;a,  it  could  not  be  confirmed  using  the  same  data.  J.P. 
Campbell  et  al.  (1990)  described  their  LISREL  analyses  using  these  data  as 
"quasi"  confirmatory. 

The  evaluation  of  the  CVI  first-tour  performance  model  using  data  from 
an  independent  sample  can,  however,  be  considered  confirmatory.  In  this 
chapter,  we  report  results  of  analyses  aimed  at  confirming  the  CVI  model, 
using  first-tour  performance  data  collected  from  the  Project  A/Career  Force 
Longitudinal  Validation  (LVI)  sample.  Additionally,  results  of  comparative 
analyses  aimed  at  evaluating  more  parsimonious  models  of  first-tour 
performance  are  reported.  These  analyses  were  also  conducted  using  the  LVI 
data.  Finally,  the  scoring  procedures  used  to  create  the  LVI  criterion 
construct  scores  used  in  the  validation  analyses  reported  in  Chapter  7  are 
described. 


A  CONFIRMATORY  TEST 
Sample 

The  CVI  model  was  developed  using  data  from  the  nine  MOS,  designated 
Batch  A,  for  which  a  full  set  of  criterion  measures  had  been  developed  (C.H. 
Campbell  et  al.,  1990),  In  the  LVI  sample,  there  are  10  Batch  A  MOS, 
including  the  nine  from  the  CVI  sample  plus  MOS  19K.  (Note  that  the  MOS 
designation  of  Motor  Transport  Operators  has  been  changed  from  64C  in  the  CVI 
sample  to  88M  in  the  LVI  sample.)  All  soldiers  in  the  LVI  sample  with 
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1. 


Core  Technical  Proficiency  (CTP) 

This  performance  construct  represents  the  proficiency  with  which  the 
soldier  performs  the  tasks  that  are  "central"  to  the  MOS.  The  tasks 
represent  the  core  of  the  job  and  they  are  the  primary  definers  of  the 
MOS.  For  example,  the  first-tour  Armor  Crewman  starts  and  stops  the 
tank  engines;  prepare  the  loader's  station;  loads  and  unloads  the  main 
gun;  boresights  the  M60A3;  engages  targets  with  the  main  gun;  and 
performs  misfire  procedures.  This  performance  construct  does  not 
include  the  individual's  willingness  to  perform  the  task  or  the  degree 
to  which  the  individual  can  coordinate  efforts  with  others.  It  refers 
to  how  well  the  individual  can  execute  the  core  technical  tasks  the  job 
requires,  given  a  willingness  to  do  so. 

2.  General  Soldiering  Proficienr_v  (GSP) 

In  .fdition  to  the  core  technical  content  specific  to  an  MOS, 
individuals  in  every  MOS  also  are  responsible  for  being  able  to  perform 
a  variety  of  general  soldiering  tasks  --  for  example,  determines  grid 
coord ina  js  on  military  maps;  puts  on,  wears,  and  removes  Ml 7  series 
protective  mask  with  hood;  determines  a  magnetic  azimuth  using  a 
compass;  collects/reports  information;  and  recognizes  and  identifies 
friendly  and  threat  aircraft.  Performance  on  this  construct  represents 
overall  proficiency  on  these  general  soldiering  tasks.  Again,  it  refers 
to  how  well  the  individual  can  execute  general  soldiering  tasks,  given  a 
wi  1 1  ingness  to  do  so. 

3.  Effort  and  Leadership  ^ELS) 

This  performance  construct  reflects  the  degree  to  which  the  individual 
exerts  effort  over  the  full  range  of  job  tasks,  perseveres  under  adverse 
or  dangerous  conditions,  and  demonstrates  leadership  and  support  toward 
peers.  That  is,  can  the  individual  be  counted  on  to  carry  out  assigned 
tasks,  even  under  adverse  conditions,  to  exercise  good  judgment,  a. id  to 
be  generally  dependable  and  proficient?  While  appropriate  knowledges 
and  skills  are  necessary  for  successful  performance,  this  construct  is 
meant  only  to  reflect  the  individual's  willingness  to  do  the  job 
required  and  to  be  cooperative  and  supportive  with  other  soldiers. 

4.  Maintaining  Personal  Discipline  (MPO) 

This  performance  construct  reflects  the  degree  to  which  the  individual 
adheres  to  Army  regulations  and  traditions,  exercises  personal  self- 
control,  demonstrates  integrity  in  day-to-day  behavior,  and  does  not 
create  disciplinary  problems.  People  who  rank  high  on  this  construct 
show  a  commitment  to  high  standards  of  personal  conduct. 

5.  Physical  Fitness  and  Military  Bearings  (PFB) 

This  performance  construct  represents  the  degree  to  which  the  individual 
maintains  an  appropriate  military  appearance  and  bearing  and  stays  in 
good  physical  condition. 


Figure  6.1.  Definitions  of  the  job  performance  constructs. 
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complete  post-imputation  performance  data  (see  Chapter  5)  are  included  in  the 
current  analyses.  The  number  of  these  soldiers  within  each  MOS  is  as  follows: 


IIB 

Infantryman 

896 

13B 

Cannon  Crewman 

801 

19E 

M60  Armor  Crewman 

■  241 

19K 

Ml  Armor  Crewman 

7P 

31C 

Single  Channel  Radio  Operator 

4c  ; 

63B 

Light-Wheel  Vehicle  Mechanic 

721 

71L 

Administrative  Specialist 

622 

88M 

Motor  Transport  Operator 

662 

91A 

Medical  Specialist 

801 

95B 

Mi  1 itary  Pol  ice 

451 

Measures 


The  complete  array  of  first-tour  performance  measures  administered  to 
the  LVI  sample  was  described  in  Chapter  5.  This  array  is  listed  in  Figure 
6.2.  There  are  two  primary  differences  between  the  array  of  first-tour 
performance  measures  administered  to  the  LVI  sample  and  that  administered  to 
the  CVI  sample.  First,  unlike  soldiers  in  the  CVI  sample,  soldiers  in  the  LV 
sample  were  not  administered  MOS-specific  written  tests  of  school  knowledge 
during  first-tour  performance  measurement  (many  were,  however,  administered 
those  tests  during  End-of-Traming  testing;  see  Chapter  3).  Second,  only 
males  in  the  LVI  sample  were  rated  with  the  Combat  Performance  Prediction 
scales.  So  as  not  to  exclude  females,  scores  from  these  scales  were  not 
included  in  the  current  analyses. 

Chapter  4  described  how  each  of  the  major  sets  of  performance  measures 
was  reduced  from  a  large  number  of  item,  task,  or  individual  scale  scores  to  a 
smaller  set  of  factor  or  category  scores.  The  results  of  this  first  level  of 
aggregation  have  been  referred  to  as  the  "basic"  array  of  criterion  scores, 
summarized  in  Table  6.1.  These  are  the  scores  that  are  used  in  the  analyses 
described  below.  For  purposes  of  comparison,  the  complete  array  of  basic 
first-tour  scores  identified  for  the  CV  sample  is  provided  in  Table  6.2. 

Despite  the  reduced  set  of  measures  administereo  to  the  LV  sample,  there 
is  a  great  deal  of  correspondence  between  the  basic  scores  created  for  the  two 
samples.  In  both  samples,  basic  scores  for  the  hands-on  tests  and  written  job 
knowledge  tests  were  created  using  the  CVBITS  (Communications,  Vehicle 
Maintenance,  Basic  Military  Skills,  identification  of  Friendly/Enemy  Aircraft 
and  Vehicles,  Technical  Skills,  and  Safety/Survival)  classification  scheme, 
which  was  developed  using  a  combination  of  expert  judgment  and  exploratory 
factor  analysis  (this  development  is  described  in  J.P.  Campbell,  1987).  Also, 
the  three  Army-  '^e  behavioral ly  anchored  rating  scales  (BARS)  composite 
scores  (Effort  Leadership,  Maintaining  Personal  Discipline,  and  Physical 
Fitness  and  Military  Bearing),  the  single  Overall  Effectiveness  rating,  and 
the  four  archival/administrative  indices  (Awards/Certificates,  Physical 
Readiness,  Articles  15,  and  Promotion  Rate)  are  common  dC»'oss  the  CVI  and  LVI 
samples . 

On  the  other  hand,  as  described  above,  soldiers  in  the  LVI  sample  did 
not  receive  basic  scores  based  on  the  written  tests  of  school  knowledge,  nor 
were  basic  scores  computed  using  the  Combat  Performance  Prediction  scales. 
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Finally,  the  two  basic  scores  (Core  Responsibilities  and  Peripheral 
Responsibilities)  derivei  from  the  MOS-specific  BARS  for  the  CVI  sample  were 
replaced  with  a  single  MOb-specific  BARS  composite  for  the  LVI  sample. 


•  Ten  behavioral ly  anchored  rating  scales  designed  to  measure  factors  of 
non- job-specific  performance  (e.g.,  providing  peer  leadership  and 
support,  maintaining  equipment,  self-discipline). 

•  Single  scale  rating  of  overall  job  performance. 

•  Single  scale  rating  of  NCO  ,.ctentia1. 

•  A  40-item  summated  rating  scale  for  the  prediction  of  combat 
performance. 

•  Four  performance  indicators  from  administrative  records,  the  first  three 
obtained  via  self-report  and  the  last  from  computerized  records. 

Total  number  of  awards  and  letters  of  commendation. 

Physical  fitness  qualification. 

Number  of  disciplinary  infractions. 

Promotion  rate  (in  deviation  units). 

•  Job-sample  (hands-on)  test  of  MOS-specific  task  proficiency. 

Individual  is  tested  on  each  of  15  major  tasks  in  an  MOS. 

•  Paper-and-penci 1  tests  designed  to  measure  task-specific  job  knowledge. 
Individual  is  scored  on  150-200  multiple-choice  items  representing  30 
major  job  tasks.  Fifteen  of  the  tasks  were  also  measured  hands-on. 

•  MOS-specific  behavioral ly  anchored  rating  scales.  From  6  to  12  BARS 
were  developed  for  each  MOS  to  represent  the  major  factors  that 
constituted  job-specific  technical  and  task  proficiency. 


Figure  6.2.  First-tour  performance  measures  administered  to  soldiers  in 
Longitudinal  Validation  sample  Batch  AMOS. 


Altogether,  the  LVI  first-tour  performance  measures  were  reduced  to  20 
basic  scores.  However,  because  MOS  differ  in  their  task  content,  not  all  20 
variables  were  scored  in  each  MOS,  and  there  was  some  slight  variation  in  the 
number  of  variables  used  in  the  subsequent  analyses.  (This  was  also  the  case 
for  the  CVI  data,  which  are  shown  in  Campbell,  1988.)  Means,  standard 
deviations,  and  intercorrelations  among  the  variables  for  each  of  the  10  jobs 
are  shown  in  Appendix  F. 
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Table  6.1 


LVI  Sample:  Twenty  Basic  Criterion  Scores  Obtained  by  Aggregating  Individual 
Rating  Scales,  Job  Sample  Tasks,  Knowledge  Test  Items,  and  Personnel  File 
Items  _  _  _ 


1.  Overall  Effectiveness  rating 

2.  Army-Wide  BARS:  Effort  and  leadership  factor 

3.  Army-Wide  BARS:  Personal  discipline  factor 

4.  Army-Wide  BARS:  Physical  fitness  and  military  bearing  factor 

5.  MOS-Specific  BARS:  Job-specific  factor 

6.  Personnel  File:  Awards  and  certificates 

7.  Personnel  File:  Physical  readiness  test  score 

8.  Personnel  File:  Articles  15/flag  actions 

9.  Personnel  File:  Promotion  rate  deviation  score 

10.  Hands-On  Test:  Core  Technical  (MOS-specif ic)  score 

11.  Hands-On  Test:  Communications  score 

12.  Hands-On  Test:  Vehicle  operation  and  maintenance  score 

13.  Hands-On  Test:  Basic  skills  (general  soldiering)  score 

14.  Hands-On  Test:  Safety  and  survival  score 

15.  Job  Knowledge  Test:  Core  Technical  (MOS-specif ic)  score 

16.  Job  Knowledge  Test:  Communications  score 

17.  Job  Knowledge  Test:  Vehicle  operation  and  maintenance  score 

18.  Job  Knowledge  Test:  Basic  skills  (general  soldiering)  score 

19.  Job  Knowledge  Test:  Identifying  target/threat  vehicles  and  aircraft 

20.  Job  Knowledge  Test:  Safety  and  survival  score 
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Table  6.2 


CVI  Sample:  Twenty-Nine  Basic  Criterion  Scores  Obtained  by  Aggregating 
Individual  Rating  Scales,  Job  Sample  Tasks,  Knowledge  Test  Items,  and 
Personnel  File  Items 


1.  Overall  Effectiveness  rating 

2.  Army-Wide  BARS:  Effort  and  leadership  factor 

3.  Army-Wide  BARS;  Personal  discipline  factor 

4.  Army-Wide  BARS:  Physical  fitness  and  military  bearing  factor 

5.  MOS-Specific  BARS:  Core  responsibilities  factor 

6.  MOS-Specific  BARS:  Peripheral  responsibilities  factor 


7.  Combat  Rating  scales;  Performing  well  under  adverse  conditions  factor 

8.  Combat  Rating  scales:  Avoiding  mistakes  factor 


9. 

10. 

11. 

12. 


Personnel  File: 
Personnel  File: 
Personnel  File: 
Personnel  File: 


Awards  and  certificates 
Physical  readiness  test  score 
Articles  15/flag  actions 
Promotion  rate  deviation  score 


13.  Hands-On  Test: 

14.  Hands-On  Test: 

15.  Hands-On  Test: 

16.  Hands-On  Test: 

17.  Hands-On  Test: 


Core  Technical  (MOS-specif ic)  score 
Communications  score 
Vehicle  operation  and  maintenance  score 
Basic  skills  (general  soldiering)  score 
Safety  and  survival  score 


18. 

19. 

20. 
21. 
22. 


Job  Knowledge  Test; 
Job  Knowledge  Test: 
Job  Knowledge  Test; 
Joo  Knowledge  Test; 
Job  Knowledge  Test: 


Core  Technical  (MOS-specif ic)  score 
Communications  score 
Vehicle  operation  and  maintenance  score 
Basic  skills  (general  soldiering)  score 
Identifying  target/threat  vehicles  and  aircraft 


23.  School  Knowledge  Test: 

24.  School  Knowledge  Test: 

25.  School  Knowledge  Test: 

26.  School  Knowledge  Test: 

27.  School  Knowledge  Test: 

28.  School  Knowledge  Test: 

29.  School  Knowledge  Test: 


Safety  and  survival  score 

Core  Technical  (MOS-specif ic)  score 

Communications  score 

Vehicle  operation  and  maintenance  score 

Basic  skills  (general  soldiering)  score 

Identifying  target/threat  vehicles  and  aircraft 

Safety  and  survival  score 
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Analyses 


Confirmatory  factor-analytic  techniques  were  applied  to  each  MOS 
individually,  using  LISREL  VII  (JOreskog  &  Sdrbom,  1989).  Several  specific 
models  were  evaluated.  The  general  model  considered  here  is  defined  by  the 
following  equation; 

y  =  Ayii  +  €  , 

where  y  are  the  observed  variable,  Ay  is  the  matrix  of  loadings  of  the 
observed  variables  on  the  unobserved  variables  (factors),  q,  and  €  are  whe 
error  terms  or  estimates  of  unique  variance  for  each  observed  variable. 

The  covariances  among  the  unobserved  variables  or  factors  are 
represented  by  the  matrix.  The  diagonal  vlements  of  the  <l>  matrix  are  fixed 
to  one  in  this  analysis,  so  that  the  element.'  are  actually  the  correlations 
among  the  unobserved  variables.  In  all  cases,  tne  model  specified  that  the 
correlation  among  the  two  method  factors  and  those  between  the  method  factors 
and  the  performance  factors  should  be  zero.  This  specification  "effectively 
defined  the  method  factor  as  that  portion  of  the  common  variance  among 
measures  from  the  same  method  that  was  not  predictable  from  (i.e.,  corre’ctcd 
with)  any  of  the  other  related  factor  or  performance  construct  scores"  (J.P. 
Campbell  et  al.,  1990,  p.  323). 

Parameters  estimated  for  each  MOS  were  the  loadings  of  the  observed 
variables  on  the  specified  common  factors,  the  unique  variances  or  errors  for 
each  observed  variable,  and  the  correlations  among  the  unobserved  variables. 
The  fit  of  the  model  was  assessed  for  each  MOS  using  the  chi-square  statistic 
and  the  root  mean-square  residual  (RMSR).  The  chi-square  statistic  indicates 
whether  the  matrix  of  correlations  among  the  variables  reproduced  using  the 
model  is  different  from  the  original  correlation  matrix.  The  RMSR,  similarly, 
summarizes  the  differences  between  entries  in  the  reproduced  matrix  and  the 
original  matrix  (Bollen,  1989). 

The  first  model  fit  to  the  LVI  data  was  a  five-factor  model  correspond¬ 
ing  to  that  developed  using  CVI  data.  As  in  the  CVI  analyses  (J.P.  Campbell 
et  al.,  1990),  out-of-range  values  were  encountered  in  a  number  of  MOS  for  the 
correlations  among  the  factors,  for  the  unique  variances,  and/or  for  the 
factor  loadings.  In  the  CVI  analyses,  this  difficulty  was  resolved  by 
setting,  for  each  MOS,  the  estimates  of  unique  variance  equal  to  one  minus  the 
squared  multiple  correlation  of  the  corresponding  variable.  When  out-of  range 
values  persisted,  the  estimate  of  unique  variance  was  reset  to  .05. 

For  the  current  LVI  analyses,  only  the  second  step  was  taken;  that  is, 
for  those  variables  for  which  the  initial  estimate  of  the  uniqueness 
coefficient  was  negative,  the  estimates  were  set  at  .05,  and  the  mooel  was 
i^er'in.  Thus,  the  uniqueness  coefficient  for  the  Physical  Fitness  and  Military 
Bearing  rating  was  set  to  .05  for  each  of  the  10  MOS,  as  was  the  uniqueness 
coefficient  for  the  Maintaining  Personal  Discipline  rating  for  IIB,  19E,  and 
88M.  These  actions  had  the  desired  effect  of  making  the  uniqueness  matrix  for 
each  MOS  positive  definite  and  bringing  the  factor  correlations  and  factor 
loadings  back  into  range. 

Note  that  it  was  necessary  to  set  the  uniqueness  estimates  for  the 
Physical  Fitness  and  Military  Bearing  rating  to  .05  in  both  the  CVI  and  LVI 
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analyses.  Thii  is  perhaps  due  to  an  identification  problem  in  the  model, 
caused  by  the  far  that  th^  "’FB  performance  factor  is  defined  by  only  two 
variables,  one  o,  which  (ti.^  Physical  Readiness  test  score)  is  only  marginally 
related  to  the  other  variables  in  the  .nodal. 

After  the  fit  of  the  five-factoi^  model  was  assessed  in  each  MOS,  four 
reduced  models  (all  nested  within  the  five- factor  model)  were  examined.  These 
models  are  described  be  low.  As  with  the  five-factor  model,  the  fit  of  the 
reduced  models  was  assessed  using  the  chi-square  statistic  and  the  root  mean- 
square  residual.  Also,  to  maintain  the  nested  structure  of  the  models,  the 
uniqueness  estimates  that  were  set  to  .05  for  the  five-factor  models  were  set 
to  ,05  for  each  of  the  reduced  models. 

Finally,  as  had  been  done  in  the  original  CVI  a.  ■■.lyses,  the  five-factor 
model  ».as  applied  to  the  Baton  A  MOS  simultaneously  (using  LISREL's 
multigroups  option).  This  model  constrained  the  following  to  be  invariant 
across  joos:  (c)  the  correlations  among  performance  factors,  (b)  the  loadings 
of  all  the  Army-.vide  measures  on  the  performance  factors  and  on  the  rating 
method  factor,  (c)  the  loadings  of  the  MOS-specific  BARS  scores  on  the  rating 
method  factor,  and  (d)  the  uniqueness  coefficients  of  the  Army-wide  measures. 
As  described  above,  the  unique  variance  for  the  Physical  Fitness  and  Military 
Bearing  rating  was  fixed  to  .05  for  all  10  MOS,  as  was  the  unique  variance  for 
the  Maintaining  Personal  Discipline  rating  for  IIB,  I9E,  and  88M. 

Results 

Confirmatory  factor  Analyses  Within  HOS 

Five-factor  model.  The  resulting  factor  loading  estimates,  uniqueness 
estimates  and  factor  intercorrelution  estimates  for  the  five-factor  model,  as 
fit  to  each  Batch  A  job  in  the  LV  sample,  are  shown  in  Tables  6.3,  6.4,  and 
6.5.  As  previously  described,  this  model  consists  of  five  performance  factors 
(CTP,  GSP,  ELS,  MPD,  and  PFB)  and  two  method  factors  (Ratings  and  Written). 

Note  that  all  of  the  CVBITS  variables  (from  the  hands-on  and  job 
knowledge  tests)  for  MOS  IIB  were  specified  tO  load  on  the  General  Soldiering 
Proficiency  performance  factor  (thus  resulting  in  only  a  four-factor  model  for 
this  MOS).  These  specifications  were  made  because  the  IIB  job  is  the  basic 
infantry  position,  consisting  of  tasks  that  soldiers  in  all  MOS  are  expected 
to  be  able  to  perform.  However,  because  these  common  tasks  form  the 
"technical"  component  of  the  infantry  MOS,  this  factor  was  treated  as  the  Core 
Technical  Proficiency  performance  factor  in  CVI  validation  analyses  (and  is 
treated  as  such  in  the  LVI  validation  analyses  reported  in  Chapter  7). 

For  comparison  purposes,  the  corresponding  results  for  the  five-factor 
model  as  applied  to  the  CVI  data  are  reported  in  Tables  6.6,  6.7,  and  6.8.  To 
make  the  comparison  as  interpretable  as  possible,  we  computed  these  results 
using  exactly  the  same  procedures  described  above  for  the  LVI  analyses.  Thus, 
these  results  are  different  fr.om  those  previously  reported  in  J.P.  Campbell  et 
al.  (1990)  in  that  they  ’nclude  only  those  variables  (i.e.,  basic  scores)  that 
were  also  a.'^’iable  for  the  LVI  analyses.  Also,  the  uniqunness  estimates  were 
fixed  to  .0.0  frr  those  variables  whose  initial  estimates  were  out  of  range 
(i.e.,  negative);  for  all  other  variables,  the  uniqueness  estimate  was 
estimated  using  LISRhL 
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Table  6.3 


Factor  Loadings  for  LVI  Five-Factor  Perfonsance  Model:  Separate  Model  for 
Each  Job 


MOS 


Factor/Score^ 

IIB 

136 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

Core  Technical 

HO  Technical  Skill 

- 

.57 

.49 

52 

.62 

.60 

.68 

.29 

.63 

.43 

JK  Technical  Skill 

- 

.59 

.64 

53 

.63 

.51 

.68 

.52 

.55 

.43 

General  Soldiering 

HO  Basic  Skill 

.67 

.71 

.45 

.58 

.48 

.38 

.67 

.72 

.57 

.51 

hO  Safety 

.48 

.31 

.30 

.43 

.39 

.41 

.34 

.48 

.60 

.37 

HO  Communication 

.20 

.24 

.48 

.38 

.55 

- 

- 

- 

- 

- 

HO  Vehicle 

- 

- 

- 

- 

.38 

.38 

- 

b 

- 

.34 

JK  Basic  Skill 

.69 

.51 

.62 

.43 

.45 

.53 

.71 

.58 

.66 

.54 

JK  Safety 

.53 

.43 

.49 

.47 

.39 

.54 

.45 

.49 

.60 

.43 

JK  Communication 

.25 

.22 

.63 

.41 

.52 

- 

- 

> 

- 

.50 

JK  Vehicle 

- 

- 

- 

- 

.30 

.42 

b 

.34 

.07 

JK  Identify 

.30 

.17 

.43 

.22 

.17 

- 

- 

.06 

.24 

.19 

Effort/Leadership 

Effort/Lead  Rating 

.73 

.68 

.70 

.59 

.53 

.62 

.67 

.61 

.62 

.50 

MOS  Rating 

.73 

.61 

.62 

.60 

.41 

.55 

.51 

.54 

.53 

.39 

Awards/Cert i f icate 

.27 

.36 

.19 

.28 

.32 

.24 

.26 

.24 

.37 

.27 

Overall  Rating 

.63 

.47 

.47 

.50 

.16 

.46 

.61 

.55 

,46 

.20 

Discipl ine 

Discipline  Rating 

.51 

.46 

.51 

.38 

.46 

.47 

.49 

.40 

.47 

.40 

Articles  15 

-54 

-.60 

-.67 

-.55 

-.57 

-.51 

-.45 

-.50 

-.52 

-.56 

Promotion  Rate 

.71 

.67 

.67 

.80 

.56 

.65 

.56 

.78 

.65 

.63 

Overal  1  Rating 

.11 

.19 

19 

.11 

.31 

.16 

.03 

.07 

.10 

.31 

F  itness/Bearing 

Fitness  Rating 

.86 

.88 

80 

.81 

.77 

.30 

.87 

.83 

.82 

.73 

Physical  Readiness 

.23 

.26 

28 

.39 

.38 

.29 

.39 

.24 

.46 

.45 

Ratings  Method 

Effort/Lead  Rating 

.57 

.60 

.60 

.70 

.74 

.66 

.65 

.69 

.67 

.77 

Discipline  Rating 

.80 

.67 

.76 

.80 

.65 

.74 

.72 

.86 

.70 

.59 

F itness  Rating 

.38 

.37 

.45 

.aq 

.  54 

.51 

.41 

.48 

.49 

.53 

Overal  1  Rating 

.54 

.59 

61 

•  6^ 

.71 

.62 

.60 

.66 

.68 

.77 

MOS  Rating 

.51 

.44 

49 

.60 

.65 

.64 

.67 

.61 

.58 

.71 

Written  Method 

JK  Technical  Skill 

- 

.49 

.35 

.56 

.60 

.62 

.34 

.64 

.72 

.68 

JK  Basic  Sk i 11 

.54 

.63 

.57 

.64 

.49 

.44 

.30 

.53 

.34 

.61 

JK  Safety 

.58 

.57 

.:6 

.52 

.55 

.40 

.40 

.57 

.60 

.61 

JK  Communication 

.34 

.13 

.38 

.33 

.56 

- 

- 

• 

- 

.52 

JK  Vehicle 

- 

- 

- 

- 

.15 

.55 

b 

.17 

.19 

JK  Identify 

.31 

.27 

.19 

.34 

.13 

- 

,11 

.12 

.16 

Note.  Dashes  indicate  the  variable  was  not  scored  in  that  MOS. 

^  H0=Hands-0n;  JK=Job  Knowledge  Test;  MOS=Job~Specif ic  Ratings. 

Vcnicle  content  was  merged  into  the  Core  Technical  factor  for  MOS  88M. 
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Table  6.4 

Uniqueness  Estimates  for  LVI  Five-Factor  Performarcs  Model:  Separate  Model 
for  Each  Job 


m 


Score* 

IIB 

136 

l''E 

19K 

31C 

63B 

71 L 

88M 

91A 

95B 

HO  Technical  Skill 

.59 

.61 

.68 

.46 

.53 

.45 

.85 

.53 

.72 

HO  Basic  Skill 

.50 

.47 

.71 

.59 

.62 

.78 

.52 

.44 

.59 

.66 

HO  Safety 

.71 

.85 

.72 

.76 

.73 

.77 

.80 

.70 

.59 

.76 

HO  Communicalion 

.92 

.90 

.68 

.75 

.64 

- 

- 

- 

- 

- 

HO  Vehicle 

- 

- 

- 

- 

.73 

.76 

- 

b 

- 

.80 

JK  Technical  Skill 

.31 

.36 

.35 

.11 

.28 

.38 

.28 

.11 

.30 

JK  Basic  Skill 

.18 

.25 

.23 

.33 

.35 

.48 

.38 

.31 

.37 

.28 

JK  Safety 

.35 

.40 

.34 

.45 

.40 

.48 

.60 

.35 

.24 

.38 

JK  Communication 

.79 

.88 

.38 

.61 

.34 

- 

. 

. 

- 

.42 

JK  Vehicle 

- 

- 

- 

- 

.73 

.45 

- 

b 

.77 

.86 

JK  Identify 

.76 

.87 

.65 

.78 

.86 

- 

- 

.93 

.90 

.86 

Effort/Lead  Rating 

.08 

.10 

.08 

.09 

.07 

.08 

.10 

.09 

.08 

.07 

Discipline  Rating 

,05 

.23 

.05 

.16 

.23 

.15 

.72 

.05 

.19 

.30 

Fitness  Rating 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

Overall  Rating 

.15 

.19 

.16 

.16 

.19 

.21 

.21 

.17 

.18 

.09 

MOS  Rating 

.16 

.33 

.24 

.22 

.27 

.19 

.26 

.25 

.28 

.22 

Awards/Certificate 

.89 

.81 

.84 

.87 

.78 

.84 

.84 

.75 

.78 

.81 

Articles  15 

,63 

.58 

.36 

-.65 

.60 

.66 

.76 

.64 

.63 

.59 

Physical  Readiness 

.84 

.85 

.79 

.79 

.78 

.87 

.77 

.85 

.72 

.71 

Promotion  Rate 

.45 

.45 

.42 

.30 

.63 

.53 

.65 

.35 

.53 

.56 

Note.  Dashes  indicate 

the 

variab 

le  was 

not 

scored 

in 

that  MO 

r 

j  - 

^  H0=Han'Js-0n;  JK=Job  Knowledge  Test;  MOS=Job-Specif ic  Ratings. 

Vehicle  content  was  merged  into  the  Core  Technical  factor  for  MOS  88M. 
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Table  6.5 


Factor  Intercorrelations  foj  LVI  Five-Factor  Performance  Model;  Separate 
Model  for  Each  Job 


MOS 


First  Second 


Factor 

Factor 

IIB 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

Core 

General  Soldier 

a 

.84 

.84 

.81 

.83 

.80 

.75 

.86 

.87 

.78 

Technical 

Effort/Lead 

a 

.51 

.32 

.40 

.44 

.50 

.45 

.35 

.51 

.42 

Discipline 

a 

.38 

.26 

.15 

.28 

.21 

.36 

.28 

.44 

.22 

Fitness 

a 

.14 

.06 

.07 

.04 

.00 

.17 

.-05 

.17 

.02 

General 

Effort/Lead 

.53 

.36 

.48 

.26 

.52 

.41 

.26 

.41 

.53 

.39 

Soldiering 

Disc  ip  1 ine 

.40 

.29 

.41 

.03 

.30 

.19 

.23 

.23 

.41 

.21 

Fitness 

.16 

-.02 

.09 

.00 

.08 

-01 

.10 

.09 

.19 

.12 

Effort/ 

Discipline 

.63 

.67 

.62 

.58 

.59 

.57 

.68 

.61 

.77 

.79 

Leadership 

Fitness 

.73 

.62 

.58 

.65 

.39 

.49 

.62 

.60 

.59 

.51 

Discipl ine 

Fitness 

.55 

.58 

.52 

.44 

.41 

.50 

.47 

.48 

.43 

.67 

No  Core  Technical  Proficiency  factor  for  MOS  IIB. 
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The  fit  of  the  five-factor  model  for  each  MOS  in  the  LVI  and  CVI  samples 
is  reported  in  Table  6.9.  Note  that  the  root  mean-square  residuals  for  the 
LVI  data  are  very  similar  to  those  for  the  CVI  data.  In  fact,  for  three  of 
the  MOS  (IIB,  13B,  and  71L),  the  RMSRs  for  the  LVI  data  are  smaller  than  those 
for  the  CVI  data.  These  results  indicate  that  the  n''del  developed  using  the 
CVI  data  does  fit  the  LVI  data  quite  well. 

Reduced  models.  As  previously  indicated,  four  reduced  models  were  also 
examined  using  the  LVI  data.  The  reason  for  examining  these  models  was  to 
determine  how  much  fit  would  be  lost  by  trying  to  achieve  a  more  parsimonious 
model  O-fi-.  ^  model  with  fewer  performance  •i’actors).  Each  of  the  reduced 
models  retained  the  two  method  factors  and  the  specification  that  these  method 
factors  be  uncorrelated  with  each  other  and  with  the  performance  factors. 

For  the  four-factor  model,  the  Core  Technical  Proficiency  and  General 
Soldiering  Proficiency  performance  factors  were  collapsed  into  a  single  "can 
do"  performance  factor.  Specifications  for  the  Effort  and  Leadership, 
Maintaining  Personal  Discipline,  and  Physical  Fitness  and  Military  Bearing 
performance  factors  were  not  altered.  The  three-factor  model  retained  the 
"can  do"  performance  factor  of  the  four-factor  model,  but  also  collapsed  the 
Effort  and  Leadership  and  Maintaining  Personal  Discipline  performance  factors 
into  a  "will  do"  performance  factor.  Once  again,  specifications  for  the 
Physical  Fitness  and  Military  Bearing  performance  factor  were  not  changed. 

For  the  two-factor  model,  the  "can  do"  performance  factor  was  retained; 
however,  the  Physical  Fitness  and  Military  Bearing  performance  factor  became 
part  of  the  "will  do"  performance  factor.  Finally,  for  the  one-factor  model, 
the  "can  do"  and  "will  do"  performance  factors  (or,  equivalently,  the  five 
original  performance  factors)  were  collapsed  into  a  single  performance  factor. 

The  chi-square  statistics  and  RMSRs,  respectively,  for  the  four  reduced 
models,  as  well  as  for  the  five-factor  model,  are  provided  in  Tables  6.10  and 
6.11.  An  examination  of  the  results  in  these  tables  indicate  that  the  four- 
and  five-factor  models  fit  the  LVI  data  well,  while  the  one-,  two-,  and  three- 
factor  models  fit  the  data  much  less  well. 

Whereas  the  average  increase  in  chi-square  was  approximately  17.7  (for  4 
degrees  of  freedom)  between  the  five-  and  four-factor  models,  it  was  51.5  (4 

degrees  of  freedom)  between  the  four-  and  three-factor  models,  104.5  (2 

degrees  of  freedom)  between  the  three-  and  two-factor  models,  and  94.7  (1 
degree  of  freedom)  between  the  two-  and  one-factor  models.  The  RMSRs  also 
increased  greatly  after  four  factors.  The  average  increase  from  the  five- 
factor  mode!  to  the  four-factor  model  was  .002,  from  the  four-  to  the  three- 

was  .015,  from  the  three-  to  the  two-  was  .027,  and  from  the  two-  to  the  one- 

was  ,042. 

These  results  suggest  that  a  model  comp»s?d  of  only  four  pei^formance 
factors  (combining  the  CTP  and  GSP  performance  factors)  and  the  method 
factors  fit  the  LVI  data  almost  as  well  as  the  original  model.  Howeve. , 
further  reductions  in  the  model  result  in  very  poor  fits  to  the  data. 
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Factor  Loadings  for  CVI  Five-Factor  Performance  Model:  Separate  Model  for 
Each  Job  _ _ _ _ _ _ 


MOS 


Factor/Score^ 

IIB 

13B 

19E 

31C 

63B 

64C 

71L 

91A 

95B 

Core  Technical 

HO  Technical  Ski  1 1 

.59 

.54 

.54 

.54 

.26 

.71 

.57 

.36 

JK  Technical  Skill 

- 

.54 

.66 

.74 

.63 

.47 

.65 

.65 

.40 

General  Soldiering 

HO  Basic  Skill 

.59 

.72 

.49 

.42 

.28 

.70 

.70 

.42 

.53 

HO  Safety 

.29 

.25 

.37 

.29 

.20 

.42 

.54 

.54 

.38 

HO  Communication 

.06 

.19 

.50 

.60 

- 

- 

- 

- 

- 

HO  Vehicle 

• 

- 

- 

.17 

.16 

b 

- 

- 

.26 

JK  Basic  bk  1  1 1 

.67 

.41 

.59 

.45 

.53 

.57 

.62 

.66 

.55 

JK  Safety 

.47 

.23 

.56 

.43 

.56 

.46 

.47 

.61 

.47 

JK  Communication 

.32 

.08 

.49 

.53 

- 

- 

- 

- 

,37 

JK  Vehicle 

- 

- 

- 

.38 

51 

b 

- 

.08 

.24 

JK  Identify 

.48 

.20 

.33 

.30 

- 

.08 

- 

.31 

.16 

Effort/Leadership 

Effort/Lead  Rating 

.74 

.56 

.71 

.66 

.58 

.63 

.64 

.69 

.56 

MOS  Rating 

.72 

.38 

.57 

.45 

.51 

.47 

.55 

.63 

.57 

Awards/Certificate 

.22 

.24 

.25 

.14 

.20 

.26 

.17 

.40 

.33 

Overall  Rating 

.65 

.32 

.24 

« 12 

.20 

.20 

.56 

.40 

.61 

Discipline 

Discipline  Rating 

.54 

.45 

.58 

.50 

.50 

.55 

.39 

.52 

.45 

Articles  15 

-.67 

-.58 

-.59 

-.54 

-.56 

-.49 

-.70 

-.49 

-.59 

Promotion  Rate 

.65 

.55 

.65 

.53 

.63 

.60 

.53 

.54 

.54 

Overall  Rating 

.11 

.26 

.41 

.49 

.34 

.37 

.04 

.23 

.04 

F  itness/Bearing 

Fitness  Rating 

.88 

.79 

.83 

.83 

.82 

.77 

.87 

.81 

.84 

Physical  Readiness 

.23 

.25 

.41 

.37 

.37 

.32 

.32 

.45 

.49 

Ratings  Method 

Effort/Lead  Rating 

.53 

.71 

.59 

.63 

.73 

.69 

.53 

.65 

.73 

Discipline  Rating 

.75 

.61 

.50 

.59 

.62 

.61 

.81 

.65 

.72 

Fitness  Rating 

.34 

.54 

.44 

.43 

.49 

.54 

.39 

.52 

.43 

Overall  Rating 

.51 

.58 

.67 

.67 

.75 

.70 

.61 

.69 

.67 

MOS  Rating 

.43 

.62 

.52 

.65 

.67 

.63 

.53 

.49 

.62 

W”itten  Method 

JK  Basic  Skill 

.65 

.62 

.59 

.47 

.43 

.64 

.39 

.24 

.63 

JK  Safety 

.53 

.69 

.50 

.61 

.23 

.59 

.43 

.59 

.49 

JK  Communication 

.32 

.16 

.32 

.46 

- 

- 

- 

- 

.44 

JK  Vehicle 

- 

- 

- 

.14 

.50 

b 

• 

.16 

.22 

JK  Identify 

.11 

.26 

.15 

.09 

• 

.12 

• 

.04 

.21 

Note.  Dashes  indicate  the  variable  was  not  scored  in  that  MOS. 


^  H0=Hands-0n;  JK=Job  Knowledge  Test;  MOS=Job-Specif ic  Ratings. 

^  Vehicle  content  was  merged  into  the  Core  Technical  factor  for  MOS  64C. 


Table  6.7 


Uniqueness  Estimates  for  CVI  Five-Factor  Performance  Model:  Separate  Model 
for  Each  Job 


MOS 


Score* 

IIB 

13B 

19E 

31C 

63B 

64C 

71L 

91A 

95B 

HO  Technical  Skill 

. 

.52 

.65 

.60 

.67 

.88 

.42 

.64 

.83 

HO  Basic  Skill 

.61 

.43 

.70 

.70 

.88 

.50 

.50 

.75 

.65 

HO  Safety 

.87 

.87 

.81 

.77 

.91 

.77 

.67 

.65 

.79 

HO  Communication 

.96 

.92 

.67 

.58 

• 

. 

• 

HO  Vehicle 

- 

- 

- 

.88 

.92 

b 

- 

- 

.83 

JK  Technical  Skill 

• 

.31 

.36 

.15 

.05 

.49 

.23 

-08 

.36 

JK  Basic  Skill 

.05 

.35 

.26 

.45 

.47 

.22 

.41 

.45 

.24 

JK  L'dfety 

.45 

.40 

.39 

.39 

.55 

.39 

.51 

.26 

.49 

JK  Coinmunication 

.76 

.90 

.59 

.43 

• 

• 

_ 

• 

.62 

JK  Vehicle 

- 

• 

.72 

.46 

b 

• 

.82 

.83 

JK  Identify 

72 

.87 

.83 

.80 

- 

.95 

- 

.82 

.84 

Effort/Lead  Rating 

.07 

.07 

.10 

.08 

.07 

.05 

.12 

.08 

.11 

Discipline  Rating 

.05 

.30 

.34 

.34 

.30 

.25 

.09 

.25 

.22 

Fitness  Rating 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

Overall  Rating 

.12 

.16 

.13 

.12 

.13 

.14 

.21 

.14 

.11 

MOS  Rating 

.18 

.35 

.36 

.24 

.21 

.31 

.33 

.34 

.24 

Awards/Certificate 

.83 

.82 

.83 

.81 

.90 

.81 

.88 

.67 

.75 

Articles  15 

.44 

,56 

.60 

.63 

.66 

.69 

.49 

.69 

.61 

Physical  Readiness 

.85 

.87 

.77 

.70 

.79 

.84 

.79 

.72 

.72 

Promotion  Rate 

.47 

.66 

.53 

.65 

.57 

.60 

.69 

.64 

.64 

Note.  Dashes  indicate 

the  variable 

was  not 

scored  in 

that 

MOS. 

^  H0*Hands-0n;  JK=Job  Knowledge;  SK*Schoo1  Knowledge. 

Vehicle  content  was  merged  into  the  Technical  factor  for  MOS  64C. 
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Table  6.8 


Factor  Intercorrelations  for  CVI  Five-Factor  Performance  Model:  Separate 
Model  for  Each  Job 


MOS 


F  irst 

Factor 

Second 

Factor 

IIB 

138 

19E 

31C 

63B 

64C 

71L 

91A 

958 

Core 

General  Soldier  ^ 

.84 

.89 

.64 

.81 

.81 

.50 

.87 

.72 

Technical 

Effort/Lead 

a 

.62 

.50 

.57 

.48 

.48 

.28 

.41 

.41 

Discipline 

a 

.27 

.33 

.37 

.22 

.36 

.20 

.43 

.28 

Fitness 

a 

.10 

.06 

.08 

-.02 

.06 

-.05 

-.01 

11 

General 

Effort/Lead 

.52 

51 

.63 

.50 

.51 

.42 

.20 

.40 

.38 

Soldiering 

Discipl  ine 

.46 

.43 

.34 

.21 

.23 

.08 

.34 

.27 

Fitness 

.15 

.06 

.09 

,10 

.05 

.06 

-.03 

.03 

.17 

Effort/ 

Discipline 

.64 

.82 

.82 

.74 

.77 

.80 

.53 

.79 

.59 

Leadership 

F  itness 

.70 

.52 

.44 

.44 

.32 

.35 

.53 

.43 

.71 

Discipl ine 

F  itness 

.45 

.58 

.43 

.59 

.53 

.45 

.44 

.40 

.45 

^  No  Core  Technical  Proficiency  factor  for  MOS  IIB. 


139 


Table  6.9 

Comparison  of  Fit  Statistics  for  CVI  and  LVI  Five-Factor  Solutions:  Separate 
Model  for  Each  Job 


MOS 

CVI 

LVI 

n 

RMSR 

Chi- 

square 

df 

n 

RMSR 

Chi- 

square 

df 

IIB" 

687 

.063 

198.1 

88 

896 

.044 

213.8 

88 

13B 

654 

.066 

218.9 

114 

801 

.059 

244.1 

114 

19E 

489 

.043 

143.0 

114 

241 

.072 

148.9 

114 

19k‘’ 

780 

.049 

236.8 

114 

31C 

349 

.060 

205.5 

148 

483 

.077 

290.4 

148 

636 

603 

.047 

129.8 

99 

721 

.065 

219.6 

99 

64C/88M‘ 

667 

.053 

140.0 

84 

662 

.057 

221.4 

84 

71L 

495 

.067 

99.8 

71 

622 

.045 

108.3 

71 

91A 

491 

.050 

162.2 

98 

801 

.056 

245.6 

98 

95B 

686 

.046 

236.7 

130 

451 

.061 

199.4 

130 

^  Fit  statistics  for  MOS  IIB  are  for  four-factor  model  (all  factors  except 
Core  Technical  Proficiency). 

^  MOS  19K  not  included  in  Concurrent  Validation  sample. 

^  MOS  64C  in  Concurrent  Validation  sam^ 'e  is  designated  as  MOS  88M  in 
Longitudinal  Validation  sample. 
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Table  6.10 


LVI  Ch1-Squares  for  Five-,  Four-,  Three-,  Two-,  and  One-Factor  Performance 
Hodels 


Five  Four  Three  Two  One 

Factors  Factors  Factors  Factors  Factor 


Chi-  Chi-  Chi-  Chi-  Chi- 


MOS 

square 

df 

square 

df 

square 

df 

square 

df 

square 

df 

IIB* 

213.8 

88 

213.8 

88 

327.8 

92 

553.6 

94 

676.6 

95 

13B 

244.1 

114 

268.4 

118 

314.4 

122 

430.1 

124 

522.6 

125 

19E 

148.9 

114 

154.6 

118 

194.8 

122 

245.8 

124 

282.1 

125 

19K 

236.8 

114 

255.6 

118 

315.1 

122 

432.5 

124 

546.1 

125 

31C 

290.4 

148 

307.6 

152 

339.4 

156 

413.0 

158 

497.7 

159 

63B 

219.6 

99 

236.7 

103 

290.3 

107 

359.8 

109 

441.6 

110 

71L 

108.3 

71 

143.9 

75 

167.2 

79 

227.8 

81 

360.6 

82 

88M 

221.4 

84 

235.0 

88 

318.2 

92 

466.7 

94 

571.5 

95 

91A 

245.6 

98 

263.8 

102 

296.9 

106 

439.2 

108 

578.6 

109 

95B 

199.4 

130 

208.2 

134 

228.2 

138 

269.2 

140 

307.3 

141 

^  Five-  and  four-factor  models  are  the  same  for  MOS  IIB. 
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Table  6.11 


LVI  Root  Mean-Square  Residuals  for 
Performance  Models 

Five-,  Four-,  Three- 

,  Two-,  and 

One-Factor 

Five 

Four 

Three 

Two 

One 

Factors 

Factors 

Factors 

Factors 

Factor 

MOS 

RMSR 

RMSR 

RMSR 

RMSR 

RMSR 

IIB* 

.044 

.044 

.064 

.092 

.134 

13B 

.059 

.063 

.071 

.070 

.114 

19E 

.072 

.072 

.098 

.141 

.212 

19K 

.049 

.049 

.069 

.091 

.134 

31C 

.077 

.084 

.119 

.147 

.163 

63B 

.065 

.066 

.074 

.079 

.104 

71L 

.045 

.053 

.054 

.078 

.150 

88M 

.057 

.057 

.072 

.100 

.150 

91A 

.056 

.056 

.058 

.122 

.159 

95B 

.061 

.060 

.071 

.095 

.123 

^  Five-  and 

four-factor 

models  are 

the  same  for  MOS  IIB. 

Confirmatory  Factor  Analyses  Across  HOS 

The  results  reported  in  Tables  6.3,  6.4,  and  6.5  indicate  that  the 
parameter  estimates  for  the  five-factor  model  were  generally  similar  across 
the  10  MOS.  The  final  step  was  to  determine  whether  the  variation  in  some  of 
these  parameters  could  be  attributed  to  sampling  variation.  To  do  this  (as 
described  earlier),  we  examined  the  fit  of  a  model  in  which  the  following  were 
invariant  across  jobs:  (a)  the  correlations  among  performance  factors,  (b) 
the  loadings  of  all  the  Army-wide  measures  on  the  performance  factors  and  on 
the  rating  method  factor,  (c)  the  loadings  of  the  MOS-specific  BARS  score  on 
the  rating  method  factor,  and  (d)  the  uniqueness  coefficients  for  the  Army¬ 
wide  measures. 

J.P.  Campbell  et  al,  (1990)  indicate  that  this  is  a  relatively 
stringent  test  of  a  common  latent  structure  across  jobs.  They  state  that  it 
is  "quite  possible  that  selectivity  differences  in  different  jobs  would  lead 
to  differences  in  the  apparent  measurement  precision  of  the  common  instruments 
or  to  differences  in  the  correlations  between  the  constructs.  This  would  tend 


142 


to  make  it  appear  that  the  different  jobs  required  different  performance 
models,  when  in  fact  they  do  not."  (p.  324) 

The  LISREL  multigroups  option  requires  that  the  number  of  observed 
variables  be  the  same  for  each  job.  However,  as  was  the  case  for  the  CV  data, 
for  virtually  every  MOS  at  least  one  of  the  CVBITS  variables  was  missing  for 
the  LVI  job  knowledge  or  hands-on  tests.  To  handle  this  problem,  the 
uniqueness  coefficients  for  these  variables  were  set  at  1.00  and  the  observed 
correlations  between  these  variables  and  all  the  other  variables  were  set  to 
zero.  It  was  thus  necessary  to  adjust  the  degrees  of  freedom  for  the  chi- 
square  statistic  by  subtracting  the  number  of  "observed"  correlations  that  we 
generated  in  this  manner.  (It  was  likewise  necessary  to  adjust  the  root  mean- 
square  residuals  for  this  analysis.) 

The  factor  loadings  estimates,  uniqueness  estimates,  and  factor 
intercorrelation  estimates  for  the  single  model  across  MOS  appear  in  Tables 
6.12,  6.13,  and  6.14,  respectively.  The  chi-square  statistic  for  this  model, 
based  on  1,332  degrees  of  freedom,  was  2,714.27.  This  result  can  be  compared 
to  the  sum  of  the  chi-square  values  (2,128.24)  and  degrees  of  freedom  p.  ,060) 
reported  in  Table  6.9  for  the  LVI  within- job  analyses.  More  specif ica’ ly,  the 
difference  between  the  chi-square  associated  with  the  single  five-factor  model 
across  MOS  and  the  sum  of  the  chi-squares  associated  with  the  10  separately 
fit  models  (i.e.,  2,714.27  -  2,128.24  =  586.03)  is  itself  distributed 
according  to  chi-square,  with  degrees  of  freedom  equal  to  the  difference 
between  the  degrees  of  freedom  associated  with  the  former  and  the  sum  of  the 
degrees  of  freedom  associated  with  the  latter  (i.e.,  1,332  -  1,060  =  272). 

These  results  indicate  that  the  fit  of  the  five-factor  model  is 
significantly  worse  when  the  parameters  listed  above  are  constrained  to  be 
equal  across  the  10  jobs.  Still,  the  root  mean-square  residuals  associated 
with  the  across-MOS  model,  reported  in  Table  6.15,  are  not  substantially 
greater  than  those  for  the  within-job  analyses  reported  in  Table  6.9.  (The 
average  RMSR  for  the  across-MOS  model  is  .0676;  the  average  for  the  within-MOS 
models  is  .0585.) 


CREATING  CRITERION  CONSTRUCT  SCORES  FOR  VALIDATION  ANALYSES 

The  analyses  reported  above  demonstrate  the  usefulness  of  the  five- 
factor,  first-tour  performance  model  in  expressing  relations  among  job 
performance  scores  for  the  LVI  data.  These  analyses  also  reveal  that  a  four- 
factor  model,  in  which  the  Core  Technical  Proficiency  and  General  Soldiering 
Proficiency  factors  are  combined,  fits  the  LVI  data  almost  as  well  as  the 
five-factor  model;  however,  condensation  beyond  the  four-factor  model  yields 
models  that  fit  the  Lyi  data  poorly. 

Performance  Factor  Scores 

To  create  criterion  construct  scores  for  use  in  validation  analyses,  we 
chose  to  base  our  scoring  procedures  on  the  five-factor  model.  Although  the 
four-factor  model  has  the  advantage  of  greater  parsimony  than  the  five-factor 
model,  the  five-factor  model  offers  the  advantage  of  corresponding  to  the 
criterion  constructs  generated  in  the  CVl  validation  analyses.  Table  6.16 
shows  the  mapping  of  the  basic  scores  on  the  five  performance  factors.  These 
scores  were  combined  as  described  below. 


Table  6.12 


Factor  Loadings  for  LVI  Five-Factor  Performance  Model:  Single  Model  Across 
All  Jobs 


MOS 

Factor/Score* 

IIB 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

Core  Technical 

HO  Technical  Skill 

- 

.55 

.50 

.51 

.61 

.56 

.67 

.31 

.60 

.41 

JK  Technical  Skill 

• 

.59 

.56 

.57 

.62 

.51 

.70 

.52 

.56 

.48 

General  Soldiering 

HO  Basic  Skill 

.64 

.70 

.47 

.53 

.49 

.39 

.63 

.72 

.53 

.55 

HO  Safety 

.47 

.30 

.31 

.39 

.40 

.41 

.32 

.47 

.57 

.34 

HO  Communication 

.17 

.23 

.43 

.42 

.52 

- 

- 

- 

- 

- 

HO  Vehicle 

- 

- 

- 

- 

.39 

.34 

- 

b 

.28 

JK  Basic  Skill 

.68 

.50 

.64 

.45 

.47 

.52 

.69 

.57 

.65 

.52 

JK  Safety 

.54 

.41 

.52 

.51 

.39 

.55 

.44 

.48 

.  50 

.44 

JK  Communication 

.24 

.19 

.52 

.41 

.50 

• 

• 

• 

• 

.41 

JK  Vehicle 

- 

- 

• 

- 

.31 

.40 

• 

b 

.37 

.14 

JK  Identify 
Effort/Leadership 

.23 

.15 

.51 

.24 

•  1  3 

- 

- 

.06 

.24 

.2/ 

Effor't/Lead  Rating*^ 

65 

.65 

.65 

.65 

.65 

.65 

.65 

.65 

.65 

.65 

MOS  Rating 

.66 

.55 

.63 

.65 

.60 

.59 

.53 

.57 

.59 

.57 

Awards/Cert' 

.26 

.26 

.26 

.26 

.26 

.26 

.26 

.26 

.26 

.26 

Overall  Rating'^ 

.56 

.56 

.  56 

.  56 

.56 

.56 

.56 

.56 

.56 

.56 

Discipl ine 

Discipline  Rating 

.44 

.44 

.44 

.44 

.44 

.44 

.44 

.44 

.44 

.44 

Articles  15' 

-.54 

-.54 

-.54 

-.54 

-.54 

-.54 

-.54 

-.54 

-.54 

-.54 

Promotion  Rate' 

.67 

.67 

.67 

.67 

.67 

.67 

.67 

.67 

.67 

.67 

Overall  Rating' 

.10 

.10 

.10 

.10 

.10 

.10 

.10 

.10 

.10 

.10 

F  itness/Bearing 

Fitness  Rating 

.82 

.82 

82 

.82 

.82 

.82 

.82 

.82 

.82 

.82 

Phys  Readiness' 

.  33 

.35 

.33 

.33 

.33 

.33 

.33 

.33 

.33 

.33 

Ratings  Method 

Eff/Ldr  Rating 

.51 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

Discipline  Rating' 

.79 

.79 

.  79 

.79 

.79 

.75 

.79 

.79 

.79 

. ,  9 

Fitness  Rating' 

.42 

.42 

.4? 

.42 

.42 

.42 

.42 

.42 

.42 

.42 

Overall  Rating' 

.59 

.59 

.59 

.59 

.59 

.59 

.59 

.59 

.55 

.59 

MOS  Rating' 

.54 

.54 

.54 

.54 

.54 

.54 

.54 

.54 

.54 

.54 

Written  Method 

JK  Tech  Skill 

- 

.48 

.44 

.54 

.61 

.60 

90 

•  W 

.65 

.74 

.63 

JK  Basic  Skill 

.51 

.63 

.50 

.64 

.49 

.45 

.30 

.54 

.33 

.64 

JK  Safety 

.56 

.57 

.48 

.51 

.54 

.40 

.43 

.58 

.58 

.60 

JK  Communication 

.33 

.14 

.47 

.33 

.56 

• 

.57 

JK  Vehicle 

- 

- 

• 

.15 

.56 

b 

.  1  fy 

.15 

JK  Identify 

30 

.27 

.06 

.34 

.14 

- 

.12 

.11 

.13 

Note.  Dashes  indicate  the  variable  was  not  scored  in  that  MOS. 

^  HO=Hands-On ;  JK-Job  Knowledge  Test;  M(JS=Job-Specif ic  Ratings. 

^  Vehicle  content  was  merged  into  the  Core  Fechniral  factor  for  MOS  88M. 
Factor  loadings  constrained  to  be  equal  across  all  MOS  for  these 
factors/measures. 


Table  6.13 


Uniqueness  Estimates  for  LVI  Five-Factor  Performance  Mofiel;  Single  Model 
Across  All  Jobs 


MOS 


Score* 

IIB 

13B 

19E 

19K 

31C 

63B 

71L 

88M 

91A 

95B 

HO  Technical  Skill 

. 

.59 

.60 

.69 

.46 

.54 

.47 

.85 

.53 

.74 

HO  Basic  Skill 

.50 

.46 

.69 

.62 

.62 

.77 

.54 

.44 

.60 

.65 

HO  Safety 

.70 

.85 

.72 

.77 

.73 

.77 

.81 

.69 

.59 

.77 

HO  Communication 

.92 

.90 

.68 

.75 

.65 

- 

. 

- 

- 

- 

HO  Vehicle 

- 

- 

- 

- 

.73 

.77 

- 

b 

- 

.80 

JK  Technical  Skill 

- 

30 

.36 

.35 

.11 

.29 

.41 

.26 

.07 

.31 

JK  Basic  Skill 

.18 

.25 

.25 

.33 

.35 

.48 

.40 

.32 

.36 

.28 

JK  Safety 

.35 

.40 

.35 

.45 

.40 

.48 

.58 

.34 

.25 

.38 

JK  Communication 

.79 

.88 

.37 

.61 

.35 

. 

_ 

_ 

.42 

JK  Vehicle 

- 

- 

- 

- 

.72 

.45 

• 

b 

.77 

.86 

JK  Identify 

.76 

.87 

.61 

.78 

.86 

- 

- 

.93 

.90 

.86 

Effort/Lead  Rating'^ 
Discipl ine  Rating^ 

.09 

.09 

.09 

.09 

.09 

.09 

.09 

.09 

.09 

.09 

.05 

.09 

.05 

.09 

.09 

.09 

.09 

.05 

.09 

.09 

Fitness  Rating^ 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

.05 

Overall  Rating" 

.16 

.16 

.15 

.16 

.16 

.16 

.16 

.16 

.16 

.16 

MOS  Rating 

.16 

.34 

.23 

.22 

.26 

.19 

.27 

.2.5 

.28 

.21 

Awards/Cert  if  icate*^ 

.83 

.83 

.83 

.83 

.83 

.83 

.83 

.83 

.83 

.83 

Articles 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

.61 

Physical  Readiness‘S 

.80 

.80 

.80 

.80 

,80 

.80 

.80 

.80 

.80 

.80 

Promotion  Rate‘s 

.47 

.47 

.47 

.47 

.47 

.47 

.47 

.47 

.47 

.47 

Note:  Dashes  indicate 

the 

variable  was 

not 

scored 

in  that  MOS 

^  H0=h5nds-0n;  JK=Job  Knowledge  Test;  MOS^^Job-Specif ic  Ratings. 

^  Vehicle  content  was  merged  into  the  Core  Technical  factor  for  MOS  88M. 
Uniqueness  coefficients  constrained  to  be  equal  across  all  MOS  for  these 
measures . 
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Table  6.14 


ractor  Intercorrelations  for  LVI  Five-Factor  Performance  Model:  Single  Model 
for  All  Jobs  _  _  _ 


F  irst 

Factor 

Second 

Factor 

MOS 

IIB 

13B  19E 

19K 

31C  63B 

71L  88M 

91A 

95B 

Core 

General  Soldier 

A 

.84  .84 

.84 

.84  .84 

.84  .84 

.84 

.84 

Technical 

Effort/Lead 

a 

.43  .43 

.43 

.43  .43 

.43  .43 

.43 

.43 

Discipline 

a 

.31  .31 

.31 

.31  .31 

.31  .31 

.31 

.31 

Fitness 

a 

.10  .10 

.10 

.10  .10 

.10  .10 

.10 

.10 

Genera  1 

Effort/Lead 

.41 

.41  .41 

.41 

.41  .41 

.41  .41 

.41 

.41 

Soldier ing 

Oiscipl ine 

.28 

.28  .28 

.28 

.28  .28 

.28  .28 

.28 

.28 

F  itness 

.09 

.09  .09 

.09 

.09  .09 

.09  .09 

.09 

.09 

Effort/ 

Discipline 

.60 

.60  .60 

.60 

.60  .60 

.60  .60 

.60 

.60 

Leadership 

F  itness 

.64 

.64  .64 

.64 

.64  .64 

.64  .64 

.64 

.64 

Oiscipl ine 

F  irness 

.48 

.48  .48 

.40 

.48  .48 

.48  .48 

.48 

.48 

Note.  Correl 

aiions  constrained 

to 

be  equal 

across  all  MOS 

“  No  Core  Tecitnical  Proficiency  factor  for  MOS  IIB. 


Table  6.15 


Root  Hean-Square  Residuals  for  LVl  Five-Factor  Performance  Model:  Sane  Model 
for  Each  Job 


Root  Mean-Sauare  Residual 

MOS 

Same  Model 
for  Each  Job 

Separate  Model 
for  Each  Job* 

IIB^* 

.073 

.044 

13B 

.068 

.059 

19E 

.080 

.072 

19K 

.054 

.049 

31C 

.073 

.077 

63B 

.071 

.055 

71L 

.062 

.045 

88M 

.063 

.057 

91A 

.O'^Q 

.056 

95B 

.063 

.061 

^  See  Table  6.9. 

Root  mean-square  residual  for  MOS  IIB  is  for  four-factor  model  (all  factors 
except  Core  Technical  Proficiency). 
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Happing  of  Perfonaance  Measures  Onto  Latent  Perforaance  Factors 
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The  Core  Technical  Proficiency  construct  is  composed  of  two  components-- 
the  MOS-specific  technical  icore  from  the  hands-on  tests  and  the  MOS-specific 
technical  score  from  the  ji’b  knowledge  tests.  For  this  and  all  other 
constructs,  the  components  were  unit  weighted;  that  is,  they  were  combined  by 
first  standardizing  them  within  MOS  and  then  adding  them  together. 

The  General  Soldiering  Proficiency  construct  is  also  composed  of  two 
major  components.  The  first  component  is  operationally  defined  as  the  sum  of 
each  of  the  CVBITS  scores  (except  the  technical  score,  which  is  a  component  of 
the  Core  Technical  Proficiency  construct)  from  the  Hands-On  test.  The  second 
component  is  defined  as  the  sum  of  the  CVBITS  scores  (again,  excluding  the 
technical  score)  from  the  Job  Knowledge  test. 

The  Effort  and  L'  ^dershio  criterion  construct  is  composed  of  three 
components,  the  first  of  which  corresponds  to  the  single  rating  for  Overall 
Effectiveness.  The  second  component  is  composed  of  two  subcomponents,  both  of 
which  are  also  standardized  within  MOS.  The  first  is  one  of  the  three  factor 
scores  derived  from  the  Army-wide  BARS  scales  (i.e.,  the  Army-wide  Effort  and 
Leadership  factor)  and  consists  of  the  unit-weighted  sum  of  five  different 
scales  (Technical  Skill;  Effort;  Leadership;  Maintain  Equipment;  Self- 
Development).  The  second  subcomponent  is  the  average  of  the  MOS-specific  BARS 
r  .ing  scales.  The  third  and  final  component  is  the  administrative  measure 
identified  as  Total  Awards/Letters. 

The  Maintaining  Personal  Discipline  construct  is  composed  of  two  major 
components.  The  first  component  is  the  Maintaining  Personal  Discipline  score 
derived  from  the  Army-wide  BARS  and  consists  of  the  unit-weighted  sum  of  three 
different  scales  (Following  Regulations;  Integrity;  Self-Control).  The  second 
component  is  the  sum  of  two  standcrdized  administrative  measures,  Article 
15/Flag  Actions  and  the  Promotion  Rate  Deviation  score. 

The  fifth  criterion  construct,  Physical  Fitness  and  Military  Bearing,  is 
also  composed  of  two  major  components.  The  first  component  is  the  Physical 
Firness  and  Military  Bearing  score  derived  from  the  Army-wide  BARS  and 
consists  of  the  unit-weighted  sum  of  two  different  scales  (Military 
Appearance;  Physical  Fitness).  The  second  component  corresponds  to  the 
administrative  measure  identified  as  the  Physical  Readiness  score. 


Criterion  Residual  Scores 

As  for  the  CVI  data,  five  residual  scores,  corresponding  to  the  five 
criterion  constructs,  wero  also  created.  This  was  done  following  the 
procedures  developed  in  CVI.  First,  a  paper-and-penci 1  "methods"  factor  was 
created  by  partial ing  from  the  total  score  on  the  Job  Knowledge  test  ^hat 
variance  shared  with  all  rf  the  non-paper-and-penci  1  criterion  measures  (i.e., 
hands-on  scores,  rating  scores,  and  administrative  records).  This  residual 
was  defined  as  the  paper-and-penc i  1  method  score.  Next,  this  paper-ano-penci 1 
method  score  was  partialed  from  each  of  the  Job  Knowledge  test  scores  used  to 
Cl eate  the  Core  Technical  Proficiency  and  General  Soldiering  Proficiency 
constructs  (as  described  above).  The  resulting  "res idua  1  ized"  Job  Knowledge 
test  scu’es  were  then  added  to  the  hands-on  scores  (v/hich  were  not 
residua  ized)  to  form  residual  Core  Technical  Proficiency  and  General 
Stldiei’ing  Proficiency  scores. 
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A  similar  procedure  was  used  to  create  residual  criterion  scores  for  the 
Effort  and  Leadership,  Maintaining  Personal  Discipline,  and  Physical  Fitness 
and  Military  Bearing  constructs.  First,  a  "total"  rating  score  was  computed 
by  standardizing  and  summing  the  overall  effectiveness  rating  score,  the  three 
Army-wide  BARS  factor  scores,  and  the  average  MOS-specific  BARS  score.  Next, 
a  rating  "method"  score  was  created  by  partial ing  from  the  total  rating  score 
that  variance  associated  with  all  of  the  non-rating  criterion  measures.  The 
resulting  method  score  was  then  partialed  from  the  rating  components  of  the 
Effort  and  Leadership,  Maintaining  Personal  Discipline,  and  Physical  Fitness 
and  Military  Bearing  constructs.  Finally,  these  residual ized  rating  scores 
were  then  combined  with  the  appropriate  administrative  measures  (which  were 
not  residual  ized  in  any  way)  to  form  residual  scores  for  the  last  three 
criterion  constructs. 

Criterion  Intercorrelation.s 

The  five  "raw"  criterion  construct  scores,  the  five  residual  criterion 
construct  scores,  the  total  rating  and  job  knowledge  scores  (described  above), 
and  the  total  score  derived  from  the  hands-on  test  were  used  to  generate  a  13 
X  13  matrix  of  criterion  interco*  'lations  for  each  MCS  in  Batch  A.  The 
averages  of  these  correlatir  s  ai ,  reported  in  Table  6.17.  These  results  are 
very  similar  to  the  correlations  that  were  reported  by  J.P.  Campbell  et  al. 
(1990)  for  the  CVI  sample,  which  are  reproduced  in  Table  6.13,  although  the 
correlations  reported  for  the  CVI  results  did  not  include  the  job  knowledge 
total  score.  Note  that  the  similarity  in  correlations  occurs  despite  the  fact 
that  the  CVI  results  are  based  on  criterion  construct  scores  that  were  created 
using  the  full  array  of  basic  scores  available  for  that  sample,  and  not  just 
those  scores  used  to  create  the  construct  scores  for  the  LVI  sample. 


CONCLUDING  COMMENTS 

These  results  indicate  that  the  five-factor  model  of  first-tour  job 
performance  developed  using  data  from  the  Project  A  Concurrent  Validation 
sample  fit  the  first-:i;i.r  Longitudinal  Validation  data  to  approximately  the 
same  degree.  This  conclusion  holds  for  the  relationships  among  the  latent 
performance  factors  (as  indicated  by  the  results  of  the  LISREL  analyses)  as 
well  as  for  the  correlations  among  the  observed  criterion  construct  scores. 

The  results  also  indicate  that  a  four-factor  model  (in  which  the  Core 
Technical  Proficiency  and  General  Soloiering  Proficiency  factors  were  combined 
into  a  single  "can  do"  factor)  fit  the  LVI  data  almost  as  well  as  the  five- 
factor  model.  Whether  this  was  also  true  for  the  CVI  data  was  not  examined; 
however,  the  average  observed  correlations  between  Core  Technical  and  General 
Soldiering  were  very  similar  across  the  two  samples  (r  =  .57  for  LV  versus  r*>.53 
for  CV).  Therefore,  we  believe  that  the  four- factor  model  may  have  also  fit 
reasonably  v^ell  as  a  CVI  sample. 

Despite  the  relatively  large  relationship  between  CTP  and  GSP, 
validation  results  reported  for  the  CVI  sample  did  indicate  that  different 
equations  were  needed  to  predict  the  two  performance  constructs  (Wise, 

McHenry,  ?t  Campbell,  1990).  Furthermore,  those  results  also  indicated  that 
the  hypothesis  of  equal  prediction  equations  across  jobs  could  be  rejected  for 
the  CTP  construct  but  not  for  GSP.  Based  on  these  p’'ovious  results  and  the 
results  reported  m  this  chapter,  it  seems  justifiable  to  use  the  criterion 
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Mean  Intercorrelations  Anong  13  Sumary  Criterion  Scores  for  the  Batch  A  NOS  in  the  LVI  Sample 
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Mean  Intercorrelations  Among  12  Sunmary  Criterion  Scores  for  the  Batch  A  MOS  in  the  CVI  Sample 
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construct  scores  associated  with  all  five  factors  in  the  Longitudinal 
Validation  analyses  reported  in  Chapter  7. 

These  results  are  both  remarkable  and  promising  from  the  point  of  view 
of  theory  development  in  job  performance.  A  multidimensional  model  of  job 
performance  has  been  strongly  confirmed  in  a  new,  large  sample.  Measures  of 
job  performance  using  hands-on  tests,  written  tests,  ratings,  and  administra¬ 
tive  measures  were  administered  to  nine  different  jobs  in  two  large  cohorts; 
the  performance  model  was  developed  in  one  cohort  and  confirmed  in  the  second. 
Chapter  7  examines  the  nat  ■*  of  the  relationships  of  the  confirmed  model  of 
job  performance  in  the  secoi  cohort  with  the  same  set  of  predictor  measures 
used  in  the  first  cohort.  These  analyses  shed  additional  light  on  the 
robustness  of  the  emerging  model. 


Chapter  7 

BASIC  VALIDATION  RESULTS  FOR  THE  LVI  SAMPLE 
Scott  H.  Oppler,  Norman  G.  Peterson,  and  Teresa  Russell 


This  chapter  summarizes  the  results  of  the  validity  evaluation  of  the 
ASVAB  and  the  Project  A  Longitudinal  Validation  (LVI)  Experimental  Battery  for 
predicting  first-tour  performance  in  the  Army.  The  results  are  based  on  the 
first-tour  performance  data  collected  from  the  Project  A/Career  Force 
longitudinal  sample.  The  objectives  of  the  analyses  described  in  this  chapter 
are  as  follows: 

(1)  Compute  the  basic  validities  for  ASVAB  and  Experimental  Battery 
predictors  against  the  first-tour  performance  factors  and  selected 
individual  performance  measures. 

(2)  Compare  the  validities  of  four  alternative  sets  of  ASVAB  scores 
(nine  ASVAB  subtests  vs.  four  ASVAB  factor  composites  vs.  AFQT  vs. 
MOS-appropr iate  Aptitude  Area  composites). 

(3)  Compare  the  validities  of  three  alternative  sets  of  ABLE  scores. 

(4)  Assess  the  incremental  validities  for  the  Experimental  Battery 
predictors  over  the  four  ASVAB  factor  composites. 

(5)  Compare  the  incremental  validitie:  of  three  alternative  sets  of 
ABLE  scores. 

(6)  Compare  the  validities  and  incremental  validities  of  the 
Experimental  Battery  predictors  under  two  different  sample  editing 
strategies . 

(7)  Compare  the  validities  and  incremental  validities  of  the 
Experimental  Battery  predictors  with  the  validities  and 
incremental  validities  reported  for  CVI, 


SAMPLE 

The  results  reported  in  this  chapter  were  based  on  two  different  sample 
editing  strategies.  The  first  mirrored  the  strategy  used  in  evaluating  the 
Project  A  CVI  predictors  against  first-tour  performance.  To  be  included  in 
those  analyses,  soldiers  in  the  CVI  sample  were  required  to  have  complete  data 
for  all  of  the  Project  A  CVI  predictor  composites,  as  well  as  for  the  ASVAB 
and  each  of  the  CVI  first-tour  performance  factors.  Corresponding  to  this 
strategy,  a  validation  sample  composed  solely  of  soldiers  having  complete  data 
for  all  the  LVI  Experimental  Battery  predictors,  the  ASVAB,  and  the  LVI  first- 
tour  performance  factors  was  created  for  the  LVI  data  set.  This  sample  is 
referred  to  as  the  "listwise  deletion"  sample. 

Table  7.1  shows  the  number  of  soldiers  across  the  10  Batch  A  MOS  who 
were  able  to  meet  the  listwise  deletion  requirements.  LVI  first-tour 
performance  measures  were  administered  to  6,815  soldiers.  Following  final 
editing  of  the  data,  a  total  of  6,458  soldiers  had  complete  data  for  all 
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Table  7.1 


Hissing  Criterion  and  Predictor  Data  for  Soldiers  Administered  LVI  First-Tour 
Performance  Measures  _ 


Number  of  Soldiers: 


in  LVI  Sample  .  6,815 

who  have  complete 

LVI  criterion  data  .  6,458 

and  who  have  ASVAB 

scores  .  6,319 

and  who  were  administered 

LV  Experimental  Battery 

(either  paper-and-penci 1 

or  computer  tests)  .  4,528 

and  for  whom  no  predictor 

data  were  missing  .  3,163 


of  the  first-tour  performance  factors.  The  validation  sample  was  further 
reduced  due  to  missing  predictor  data  from  the  ASVAB  and  the  LVI  Experimental 
Battery.  Of  the  6,319  soldiers  who  had  complete  criterion  data  aind  whose 
ASVAB  scores  were  accessible,  4,528  were  administered  at  least  a  portion  of 
the  Experimental  Battery  (either  the  paper-and-penci 1  tests,  the  computer 
tests,  or  both).  Of  these,  the  total  number  of  soldiers  with  complete 
predictor  data  was  3,163. 

The  number  of  soldiers  with  complete  predictor  and  criterion  data  in 
each  MOS  is  reported  in  Table  7.2  for  both  the  CVI  and  LVI  data  sets.  With 
the  exception  of  the  73  soldiers  in  MOS  19E,  the  soldiers  in  the  right-hand 
column  of  the  table  form  the  LVI  listwise  deletion  validation  sample.  19E  was 
excluded  from  the  present  analyses  for  three  reasons.  First,  the  sample  size 
for  this  MOS  was  considerably  smaller  than  that  of  the  other  Batch  A  MOS 
(e.g.,  the  MOS  with  the  next  smallest  sample  had  172  soldiers).  Second,  the 
MOS  is  currently  being  pnased  out  of  operation.  And  third,  the  elimination  of 
19E  creates  greater  correspondence  between  the  CVI  and  LVI  samples  with 
respect  to  the  composition  of  MOS  (e.g.,  the  ratio  of  combat  to  non-combat 
MOS). 


.  In  the  alternative  sample  editing  strategy,  a  separate  validation  sample 
was  identified  for  each  set  of  predictors  in  the  Experimental  Battery  (see 
below).  More  spec  if  i  i-i.  1  ly ,  to  be  included  in  the  validation  sample  for  a 
given  predictor  set,  soldiers  were  required  to  have  complete  data  for  each  of 
the  first-tour  performance  factors,  the  ASVAB,  and  the  predictor  composites  in 
tiiat  predictor  set  only.  For  example,  a  soldier  who  had  data  for  the  complete 
set  of  ABLE  composites  (as  well  as  complete  ASVAB  and  criterion  data),  but  was 
missing  data  from  the  AVOICE  composites,  would  have  been  included  in  the 
"setwise  deletion"  sample  fcr  estimating  the  validity  of  the  former  test,  but 
not  the  latter. 


156 


Table  7.2 

Soldiers  in  CVI  and  LVI  Data  Sets  With  Complete  Predictor  and  First-Tour 
Criterion  Data,  by  MOS _ _ _ _ 


LVI  (Listwise 


MOS 

CVI  Deletion  Samp 

IIB 

Infantryman 

491 

235 

13B 

Cannon  Crewman 

464 

553 

19E' 

M60  Armor  Crewman 

394 

73 

19K 

Ml  Armor  Crewman 

— 

446 

31C 

Single  Channel  Radio  Operator 

289 

172 

63B 

Light-Wheel  Vehicle  Mechanic 

478 

406 

71L 

Administrative  Specialist 

427 

252 

88M 

Motor  Transport  Operator 

507 

221 

91A 

Medical  Specialist 

392 

535 

95B 

Ml  1  itary  Police 

597 

270 

Total 

4,039 

3,163 

*  MOS  19E  not  included  in  validity  analyses. 

There  were  two  reasons  for  creating  these  setwise  deletion  samples.  The 
first  reason  was  to  maximize  the  sample  sizes  used  in  estimating  the  validity 
of  the  Experimental  Battery  predictors.  The  number  of  soldiers  in  each  MOS 
meeting  the  setwise  deletion  requirements  for  each  predictor  set  is  reported 
in  Table  7.3.  As  can  be  seen,  the  setwise  sample  sizes  are  considerably 
larger  than  those  associated  with  the  listwise  strategy. 

Table  7.3 


Soldiers  in  LVI  Setwise  Deletion  Samples  for  Validation  of  Spatial,  Computer, 


JOB.  ABLE,  and  AVOICE  Experimental  Battery  Predictor  Composites,  by  MOS 


MOS 

Setwise 

-1^ - »  »  ^  vw. - 

Deletion  Samples 

_ _ 

Spatial 

Computer 

JOB 

ABLE 

AVOICE 

IIB 

785 

283 

720 

731 

747 

13B 

713 

670 

557 

753 

673 

19£" 

88 

86 

83 

80 

87 

19K 

548 

539 

512 

495 

527 

31C 

221 

204 

208 

200 

208 

63B 

529 

499 

498 

468 

507 

71L 

328 

302 

300 

291 

287 

88M 

279 

289 

258 

263 

257 

91A 

643 

619 

613 

597 

625 

95B 

316 

306 

307 

294 

302 

Total 

4,450 

3,797 

4,156 

4,072 

4,220 

*  MO?  ;9E  not  included  in  .a  1  id i .y  ana  I yses . 
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The  second  reason  for  using  the  setwise  strategy  stemmed  from  the  desire 
to  create  validation  samples  that  might  be  more  representative  of  the 
examinees  for  whom  test  scores  would  be  available  under  operational  testing 
conditions.  Under  the  listwise  deletion  strategy,  soldiers  were  deleted  from 
the  validation  sample  for  missing  data  from  any  of  the  tests  included  in  the 
Experimental  Battery.  In  many  instances,  these  missing  data  could  be 
attributed  to  scores  for  a  given  test  being  set  to  missing  because  the 
examinee  failed  to  pass  the  random  response  index  for  that  test,  but  not  for 
any  of  the  other  tests. 

It  is  unlikely,  however,  that  all  of  the  tests  (and,  hence,  all  of  the 
random  response  indices)  in  the  Experimental  Battery  would  be  used 
operationally  at  the  same  time.  Under  the  setwise  strategy,  any  examinee  who 
would  have  failed  the  random  response  index  for  a  test  not  administered,  but 
who  did  not  fail  the  random  response  indices  for  the  tests  that  actually  were 
administered,  would  not  be  identified  as  having  suspect  data,  (for  example, 
if  the  ABLE  was  operational  as  a  selection  device,  but  the  AVOICE  was  not,  it 
could  not  be  known  whether  a  given  examinee  would  have  passed  the  random 
response  index  associated  with  the  AVOICE.)  The  advantage  of  the  setwise 
deletion  strategy  is  that  none  of  the  examinees  removed  from  the  validation 
sample  for  a  given  test  were  excluded  solely  for  failing  the  random  response 
index  on  a  different  test  in  the  Experimental  Battery. 

As  a  final  note,  there  is  no  reason  to  expect  systematic  differences 
between  the  results  obtained  with  the  listwise  and  setwise  deletion  samples. 
However,  bacause  of  the  greater  sample  sizes  of  the  setwise  deletion  samples, 
as  well  as  the  possibly  greater  similarity  between  the  setwise  deletion 
samples  and  the  future  examinee  population,  it  is  possible  that  the  validity 
estimates  associated  with  these  samples  may  be  more  accurate  than  those 
associated  with  the  listwise  deletion  sample. 


MEASURES 

Predictors 

The  predictor  scores  used  in  these  analyses  were  derived  from  the 
operationally  administered  ASVA8  and  the  paper-and-penc i  1  and  computerized 
tests  aaministerea  in  the  Project  A  LVI  Experimental  Battery.  For  the  ASVAB, 
four  types  of  scores  were  examined.  These  scores,  listed  in  Table  7.4, 
include  the  nine  ASVAB  subtests  (of  which  the  Verbal  score  is  a  composite  of 
the  Paragraph  Comprehension  and  Word  Knowledge  subtests),  the  four  ASVAB 
factor  composite  scores,  the  AFQT,  and  the  MOS-appropriate  Aptitude  Area 
composite  scores. 

The  jcores  derived  from  the  LVI  Experimental  Battery  are  listed  in  Table 
7.5.  With  one  exception,  these  scores  are  described  in  the  Career  Force  first 
annual  report  (Campbell  &  Zook,  1990).  The  exception  concerns  the  scores 
derived  for  the  ABlE.  Note  that  three  different  sets  of  ABLE  scores  are 
listed  in  Table  7.5.  The  first  set,  labeled  the  ABLE  Rational  Composites, 
were  derived  along  with  the  other  LV  predictor  composites.  The  other  two 
sets,  labeled  ABLE-168  Composites  and  A8LE-114  Composites,  were  based  on 
results  of  factor  analyses  of  tne  ABLE  items.  Factor  Scores  (168)  were  scored 


using  168  ABLE  items,  whereas  factor  Scores  (114)  were  scored  using  only  114 
items.  See  Chapter  2  for  a  description  of  the  development  of  the  AbLc  factor 
scores. 


Table  7.4 

Four  Sets  of  ASVAB  Scores  Used  in  LVl  First-Tour  Validity  Analyses _ 

ASVAB  Subtests 

General  Science 
Arithmetic  Reasoning 

Verbal  (Paragraph  Comprehension  +  Word  Knowledge) 

Numerical  Operations 
Coding  Speed 
Auto/Shop  Information 
Mathematical  Knowledge 
Mechanical  Comprehension 
Electronic  Information 

ASVAB  Factor  Composites 

Technical  (Auto/Shop,  Mechanical  Comprehension,  Electronics  Information) 
Quantitative  (Math  Knowledge,  Arithmetic  Reasoning) 

Verbal  (word  Knowledge,  Paragraph  Comprehension,  General  Science) 

Speed  (Coding  Speed,  Number  Operations) 

AFQT 

Aptitude  Area  Composites  (1  per  MOS) 

118:  CO  (Combat) 

12B:  CO  (Combat) 

13B:  FA  (Field  Artillery) 

16S:  OF  (Operators/Foods) 

19E:  CO  (Combat) 

19K:  CO  (Combat) 

27c:  EL  (Electronics) 

29E:  EL  (Electronics) 

31C:  SC  (Siirvei  1  lance/Communications) 

518:  GM  (General  Maintenance) 

648:  ST  (Skilled  Technical) 

55B:  GM  (General  Maintenance) 

63B:  MM  (Mechanical  Maintenance) 

67N:  MM  (Mechanical  Maintenance) 

71L:  CL  (Clerical) 

76Y:  CL  (Clerical) 

88M:  OF  ( Opera  tor s /Foods  ) 

91A:  ST  (Skilled  Technical) 

948:  OF  (Operators/Foods ) 

95B:  SI  (Skilled  Technical) 

96B:  ST  (Skilled  Technical) 


Table  7.5 


Sets  of  LVI  Experimental  Battery  Predictor  Scores  Used  in  LVI  First-Tour 
Validity  Analyses  _ 


Spatial  Composite 
Spatial 

Computer  Composites 

Psychomotor 
Perceptual  Speed 
Perceptual  Accuracy 
Number  Speed  and  Accuracy 
Basic  Speed 
Basic  Accuracy 
Short-Term  Memory 
Movement  Time 

JOB  Composites 

Autonomy 

High  Expectations 
Ro..tine 

AVOICE  Composites 

Admini strat i ve 
Audiovisual  Arts 
Food  Service 
Structural /Machines 
Protective  Services 
Rugged/Cutdoors 
Social 

Skilled  Technical 


ABLE  Rational  Composites 

Achieveme;.t  Orientation 

Adjustment 

Physical  Condition 

Internal  Control 

Cooperativeness 

Oependabi 1 ity 

Leadership 

ABLE-168  Composites 

Locus  of  Control 
Cooperativeness 
Dominance 
Oependabi 1 i ty 
Physical  Condition 
Stress  olerance 
Work  Orientation 

ABLE-114  Composites 

Locus  of  Control 
Cocperat i veness 
Dominance 
Oependabi 1 ity 
Physical  Condition 
Stress  Tolerance 
Work  Orientation 


Criteria 

The  f'lrst-tour  oerformance  measures  collected  fvom  the  LVI  sample 
constitute  a  suoset  of  the  first-tour  measures  collected  from  the  CVi  sample. 
These  measures  include; 

0  Revised  Hands-On  job  sample  measures 

o  Reviseo  Job  Knowledge  tests 

0  Army-wide  BARS 

o  MOS- specific  BARS 

0  Administrative  indices  ct  performance 

These  measures  generated  a  set  of  20  basic  scores  that  were  the  basis 
for'  the  LVI  performance  modeling  analysis  reported  m  Chapter  6.  Those 
analyses  indicated  that  the  factor  model  developed  with  the  CVI  data  yielded 
an  aoequate  fit  when  applied  to  the  LVI  data.  This  model  specified  the 
existence  of  five  substantive  performance  factors  and  two  method  factors 
("written"  and  "ratings'  }.  Trie  two  methods  factors  were  defined  to  be 
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orthogonal  to  the  substantive  factors,  but  the  correlations  among  the 
substantive  factors  were  noi  so  constrained. 


The  five  substantive  factors  and  the  variables  that  are  scored  on  each 
are  listed  in  Table  7.6.  As  in  the  scoring  of  the  CVI  data,  both  a  raw  and  a 
resi'iual  score  were  created  for  each  substantive  factor.  The  residual  scores 
for  the  two  "can  do"  performance  factors  (Core  Technical  Proficiency  and 
General  Soldiering  Proficiency)  were  constructed  by  partialing  out  variance 
associated  with  the  written  method  factor,  and  the  residual  scores  for  the 
three  "will  do"  scores  were  constructed  by  removing  variance  associated  with 
the  ratings  method  factor. 

Consistent  with  the  procedures  used  for  CVI,  the  GSP  factor  scores  (raw 
or  'esidual)  created  for  soldiers  in  MOS  IIB  are  treated  as  CTP  scores  in  the 
validity  analyses.  (As  has  been  explained  elsewhere,  tasks  that  are 
considered  "general"  to  the  Army  for  soldiers  in  most  other  MOS  are  considered 
central  or  "core"  to  soldiers  in  IIB.)  In  addition  to  the  raw  and  residual 
performance  factors  and  the  two  method  factors,  total  scores  from  the  Hands-On 
and  Job  Knowledge  tests  were  also  used  in  the  validation  analyses  reported  In 
this  chapter. 


Table  7.6 

LVI  First-Tour  Performance  Factors  and  the  Basic  Criterion  Scores  That 
Define  Them 


0  Core  Technical  Proficiency  (CTP) 

Hands-On  Test  -  MOS  Specific  Tasks 
Job  Knowledge  Test  -  MOS  Specific  Tasks 

o  General  Soldiering  Proficiency  (GSP) 

Hands-On  Test  -  Common  Tasks 
Job  Knowledge  Test  -  Common  Tasks 

0  Effort  and  Leadership  (ELS) 

Admin.  Index  -  Number  of  Awards  and  Certificates 
Army-Wide  BARS  Overall  Effectiveness  Rating  Scale 
Army-Wide  BARS  Effort/Leadership  Ratings  Factor 
Average  of  MOS  BARS  Ratings  Scales 

0  Maintaining  Personal  Discipline  (HPD) 

Admin.  Index  -  Number  of  Articles  15  and  Flag  Actions 
Admin.  Index  -  Promotion  Grade  Deviation  Score 
Army-Wide  BARS  Personal  Discipline  Ratings  Factor 

0  Physical  Fitness  and  Military  Bearing  (PFB) 

Admin.  Index  -  Physical  Readiness  Score 
Army-Wide  BARS  F  i  tness/Bear ing  Ratings  Factor 
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PROCEDURE 


The  analysis  procedure  consisted  of  the  following  steps: 


A) 


Using  the  listwise  deletion  sample,  multiple  correlations  between 
each  set  of  predictor  scores  and  the  five  raw  substantive  factor 
scores,  their  five  residual  factor  scores,  the  two  method  factor 
scores,  and  the  total  scores  from  the  hands-on  and  job  knowledge 
tests  were  computed  separately  by  MOS  and  then  averaged. 

1)  As  indicated  above,  ASVAB  was  represented  by 


2) 


a)  The  nine  ASVAB  subtest  scores 

b)  The  four  ASVAB  factor  scores 

c)  The  AFQT 

d)  The  MOS-appropr late  Aptitude  Area  composite  score 

ABLE  was  represented  by  three  sets  of  scores: 

a)  The  seven  rational  scale*^^  developed  for  the 
Experimental  Battery 

b)  Seven  empirical  scales  developed  on  the  basis  of  a 
factor  ...alysis  that  retained  all  of  the  items  (and 
which  used  168  of  them) 


c)  Seven  empirical  scales  developed  by  using  the  results 
of  the  factor  analysis  to  select  the  best  items  to 
reflect  each  factor  (and  which  used  only  114  items) 

3)  Each  of  the  other  predictor  sets  was  represented  by  a  single 
set  of  scores  as  described  above  and  in  Table  7.5. 


Results  were  computed  both  with  and  without  correcting  for 
multivariate  range  restriction  (Lord  &  Novick,  1968).  Corrections 
for  range  restriction  were  made  using  the  9x9  intercorre  lat ion 
matrix  among  the  subtests  in  the  1980  Youth  Population  (OoO, 

1982).  All  results  were  adjusted  for  shrinkage  using  Rozeboom's 
( 1978)  Formula  8. 

B)  Using  the  listwise  deletion  sample,  incremental  validities  for 
ea^'h  set  of  Experimental  Battery  predictors  (e.g.,  AVOICE 
composites  or  computer  composites)  over  the  fou.-  ASVAB  factor 
composites  were  computed  against  the  same  criteria  used  to  compute 
the  validities  in  Step  A.  Once  again,  the  results  were  computed 
separately  by  MOS  and  then  averaged.  Also,  the  results  were 
computed  both  with  and  without  correcting  for  range  restriction, 
and  were  adjusted  for  shrinkage  using  the  Rozeboom  formula. 

C)  Using  the  setwise  deletion  samples,  multiple  correlations  and 
incremental  validities  (over  the  four  ASVAB  factor  composites) 
between  each  set  of  Experimental  Battei^y  predictors  and  the 
criteria  used  in  the  first  two  Steps  were  computed  separately  by 
MOS  and  then  averaged.  These  results  were  corrected  for  ranae 
restriction  and  adjusted  for  shrinkage  using  the  Rozeboom  formula. 


The  results  were  then  compared  witn  the  results  obtained  in  steps 
A  and  B  above. 

D)  Finally,  once  again  using  the  listwise  deletion  sample,  multiple 
correlations  and  incremental  validities  (over  the  four  ASVAB 
factors)  were  computed  for  each  set  of  predictors  in  the 
Experimental  Battery,  this  time  adjusting  the  results  for 
shrinkage  with  the  Claudy  (1978)  instead  of  the  Rozebocm  formula. 
This  step  was  conducted  to  allow  comparisons  between  the  first- 
tour  validity  results  associated  with  the  longitudinal  sample  and 
those  that  had  been  reported  for  the  concurrent  sample  (for  which 
only  the  Claudy  formula  was  used,  e.g.,  McHenry,  Hough,  Toquam, 
Hanson,  &  Ashworth,  1990). 


RESULTS 

Multiple  Correlations  for  ASVAB  and  LVI  Experimental  Battery 
predictors  (2a<ed  on  Listwise  Deletion  Sample) 

Multiple  correlations  for  the  four  ASVAB  factor  composites,  the  single 
spatial  composite,  the  eight  computer  com.posites,  the  three  JOB  composites, 
the  seven  ABLE  composites,  and  the  eight  AVOICE  composites  are  reported  in 
Table  7.7.  Using  tl.e  listwise  deletion  sample,  these  results  were  computed 
separately  by  MOS  and  then  averaged.  These  results  have  also  been  adjusted 
for  shrinkage  using  the  Rczeboom  formula  and  corrected  for  range  restriction. 
Results  which  have  not  been  corrected  for  range  restriction  (but  which  nave 
been  adjusted  for  shrinkage)  are  reported  in  Table  7.8. 

ihe  results  in  Table  7.7  indicate  that  the  four  ASVAB  factc''  composites 
were  the  best  set  of  predictors  for  the  raw  CTP,  GSP,  ELS,  and  MPO  performance 
factors,  the  residual  CTP,  GSP,  ELS,  and  MPO  performance  factors,  the  written 
and  ratings  method  factors,  and  the  Hands-on  and  Job  Knowledge  total  scores. 
The  spatial  composite  and  the  eignt  computer  composites  were  next  in  line, 
except  tor  MPD,  where  the  ABLE  composites  and  spatial  composite  were  next. 

The  seven  ABLE  composites  had  the  highest  level  of  validity  for  predicting  the 
raw  and  residual  PFB  factor,  with  the  ASVAB  factor  composites  second.  A 
similar  pattern  of  results  was  found  for  the  uncorrected  coefficients  reported 
in  Table  7.3. 

The  average  multiple  correlations  (corrected  and  uncorrected  for  range 
restriction,  respectively)  for  the  four  different  sets  of  ASVAB  scores  arc 
reported  in  Tables  7.9  and  7.10.  The  corrected  results  indicate  that  the  four 
ASVAB  factor  composites  consistently  had  higher  validities  than  the  other 
three  sets  of  scores,  whereas  the  AFQT  tended  to  have  the  lowest  validities. 

On  the  other  hand,  for  several  criteria,  the  uncorrected  correlations 
for  the  Aptitude  Area  composite  scores  (ar"*  sometimes  for  the  AFQT  composite) 
were  greater  than  the  uncorrected  multiple  correlations  for  the  four  ASVAB 
factors.  (It  should  be  noted  that  differences  between  the  patterns  of  the 
corrected  and  uncorrected  results  could  arise  due  to  differences  in  range 
restriction  across  the  factor  scores  and  the  Aptitude  Area  composites. 
Therefore,  where  differences  between  the  two  sets  of  results  are  reported,  we 
beli-rve  that  the  corrected  results  should  be  a'.rr  greater  enipnasis,  since 
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Table  7.7 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE: 


Results  Corrected  for  Range  Restriction 


Criterion^ 

No.  of 
MOS^ 

ASVAB 

Factors 

[4] 

Spatial 

fl] 

Computer 

[8] 

JOB 

P] 

ABLE  Comp. 
[7] 

AVOICE 

rsi 

CTP  (Raw) 

9 

62 

(13) 

57 

(11) 

47 

(16) 

29 

(13) 

21 

(09) 

38 

(08) 

GSP  (Raw) 

8 

66 

(07) 

64 

(06) 

55 

(08) 

29 

(13) 

23 

(14) 

37 

(07) 

ELS  (Raw) 

9 

37 

(12) 

32 

(08) 

29 

(15) 

18 

(14) 

13 

(11) 

17 

(15) 

MPD  (Raw) 

9 

17 

(13) 

14 

(11) 

10 

(16) 

06 

(13) 

14 

(11) 

05 

(10) 

PFB  (Raw) 

9 

16 

(06) 

10 

(04) 

07 

(07) 

06 

(06) 

27 

(07) 

05 

(09) 

CTP  (Res) 

9 

46 

(17) 

42 

(15) 

29 

(22) 

17 

(12) 

08 

(11) 

28 

(12) 

GSP  (Res) 

8 

51 

(10) 

51 

(08) 

41 

(10) 

18 

(11) 

12 

(12) 

26 

(09) 

ELS  (Res) 

9 

46 

(Ifl) 

41 

(13) 

3; 

120) 

23 

(15) 

21 

(15) 

24 

(16) 

MPD  (Res) 

9 

18 

(13) 

14 

(12) 

08 

(16) 

07 

(11) 

13 

(11) 

06 

(10) 

PFB  (Res) 

9 

20 

(10) 

12 

(08) 

09 

(11) 

07 

(06) 

28 

(10) 

09 

(11) 

Wr  i  tten 

9 

54 

(13) 

49 

(12) 

43 

(18) 

29 

(16) 

23 

(12) 

29 

(14) 

Ratings 

9 

12 

(09) 

09 

(07) 

07 

(09) 

06 

(09) 

03 

(05) 

02 

(07) 

HO-Tota 1 

9 

50 

(14) 

48 

(11) 

38 

(15) 

18 

(13) 

11 

(11) 

28 

(09) 

JK-Tota  1 

9 

71 

(08) 

65 

(07) 

58 

(10) 

36 

(14) 

31 

(08) 

41 

(08) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Decimals  omitted. 

^  CTP  =  Core  Technical  Proficiency;  GSP  =  General  Soldiering  Proficiency; 
ELS  =  Effort  and  Leadership;  MPD  =  Maintaining  Personal  Discipline; 

PFB  =  Physical  Fitness  and  Military  Bearing;  HO  =  Hands-On;  JK  =  Job 
Knowledge. 

^  Number  of  MOS  for  which  validities  were  computed. 


Table  7.8 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE  Composites,  and  AVOICE; 
Results  Uncorrected  for  Range  Restriction _ 


ASVAB 


No.  of 

Factors 

Sp 

at  ia  1 

Computer 

JOB 

ABLE 

;  Comp. 

AVOICE 

Cri 

ter  ion 

MOS" 

[4] 

[1] 

[8] 

[3] 

[7] 

[8] 

CTP 

(Raw) 

9 

41 

(09) 

37 

(08) 

20 

(12) 

06 

(06) 

02 

(03) 

16 

(11) 

GSP 

(Raw) 

a 

46 

(06) 

46 

(06) 

26 

(13) 

06 

(08) 

05 

(08) 

14 

(11) 

ELS 

(Raw) 

9 

19 

(08) 

16 

(04) 

11 

(11) 

05 

(09) 

04 

(06) 

05 

(08) 

MPO 

(Raw) 

9 

09 

(OS) 

08 

(05) 

04 

(06) 

01 

(03) 

13 

(09) 

00 

(00) 

PFB 

(Raw) 

9 

08 

(09) 

04 

(03) 

03 

(05) 

06 

(06) 

27 

(07) 

05 

(08) 

CTP 

(Res) 

9 

30 

(09) 

27 

(09) 

12 

(10) 

02 

(04) 

00 

(00) 

14 

(14) 

GSP 

(Res) 

3 

36 

(04) 

36 

(03) 

19 

(11) 

03 

(06) 

03 

(06) 

10 

(09) 

ELS 

(Res ) 

9 

26 

(12) 

24 

(05) 

12 

(10) 

06 

(08) 

07 

(09) 

09 

(06) 

MPD 

(Res) 

9 

07 

(08) 

09 

(04) 

04 

(08) 

00 

(00) 

11 

(10) 

00 

(00) 

PFB 

(Res) 

9 

10 

(09) 

05 

(04) 

03 

(06) 

04 

(05) 

29 

(09) 

06 

(10) 

Wr  i 

tten 

9 

29 

(07) 

28 

(09) 

13 

(11) 

08 

(08) 

05 

(08) 

04 

(07) 

Rat 

ings 

9 

03 

(05) 

04 

(03) 

04 

(07) 

02 

(05) 

04 

(07) 

01 

(02) 

HO- 

Tota  1 

9 

35 

(06) 

35 

(05) 

19 

(11) 

03 

(06) 

01 

(02) 

13 

(10) 

JK- 

Tota  1 

9 

48 

(09) 

45 

(08) 

26 

(11) 

11 

(09) 

08 

(10) 

15 

(10) 

Not 

e:  Adjusted  for 

shr 

inkage 

(Rozeboom 

formula  8 

). 

Numbers  in 

parentheses 

are 

standard 

dev 

iat ions 

Numbers 

in 

br  ackets 

are  t 

he  numbers  of 

predictor  scores 

enter  i 

ng 

predict 

ion 

equat 

ions 

.  De 

c  i  ma  1 

5  omitted. 

a 


Number  of  MOS  for  which  validities  were  computed. 


Table  7.9 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ASVAB  Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  MOS-Appropriate 
Aptitude  Area:  Results  Corrected  for  Range  Restriction _ 


ASVAB  ASVAB  Aptitude 

No.  of  Subtests  Factors  AFQT  Area'’ 

Cr  i  ter  ion _ MOS^ _ [9] _ [4] _ [Ij _ [  1  j 


CTP  (Raw) 

9 

61  (14) 

62  (13) 

57  (15) 

61 

(13) 

GSP  (Raw) 

8 

66  (07) 

66  (07) 

62  (08) 

64 

(08) 

ELS  (Raw) 

9 

34  (16) 

37  (12) 

34  (12i 

37 

(11) 

MPO  (Raw) 

9 

14  (15) 

17  (13) 

14  (15) 

16 

(14) 

PFB  (Raw) 

9 

10  (09) 

16  (06) 

12  (06) 

11 

(06) 

CTP  (Res) 

9 

45  (18) 

46  (1?) 

39  (19) 

46 

(17) 

(10) 

GSP  (Res) 

8 

50  (10) 

51  (10) 

45  (09) 

49 

ELS  (Res) 

9 

44  (22) 

46  (18) 

18  (13) 

43  (20) 

46 

(18) 

MPD  (Res) 

9 

13  (14) 

15  (15) 

17 

(14) 

PFB  (Res) 

9 

15  (11) 

20  (10) 

15  (11) 

14 

(12) 

Wr i tten 

9 

54  (14) 

54  (13) 

55  (12) 

53 

(14) 

Ratings 

9 

09  (10) 

12  (09) 

11  (10) 

12 

(10) 

HO-Tota 1 

Si 

49  (14) 

50  (14) 

43  (16) 

48 

(15) 

JK-Tota 1 

9 

71  (09) 

71  (08) 

69  (09) 

70 

(09) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  3).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  bracKets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Decimals  omitted. 


Number  of  MOS  for  which  validities  were  computed. 
MOS-Appropriate  Aptitude  Area  composite. 


Table  7.10 


Mean  of  Multiple  Correlations  Comput.'^d  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ASVAB  Subtests,  ASVAB  Factors,  AFQT,  and  ASVAB  HOS-Appropriate 
Aptitude  Area:  Results  Uncorrected  for  Range  Restriction _ 


Criterion 

No.  of 
MOS" 

ASVAB 

Subtests 

[9] 

ASVAB 

Factors 

[4] 

AFQT 

rn 

Aptitude 

Area° 

[1] 

CTP  (Raw) 

9 

AO  (08) 

41  (09) 

36  (09) 

41  (07) 

GSP  (Raw) 

8 

44  (08) 

46  (06) 

41  (07) 

43  (05) 

ELS  (Raw) 

9 

15  (11) 

19  (08) 

17  (06) 

20  (05) 

MPD  (Raw) 

9 

07  (07) 

09  (08) 

11  (06) 

12  (05) 

PFB  (Raw) 

9 

06  (08) 

08  (09) 

07  (05) 

06  (04) 

CTP  (Res) 

9 

28  (10) 

30  (09) 

25  (10) 

31  (08) 

GSP  (Res) 

8 

34  (05) 

36  (04) 

30  (07) 

34  (04) 

ELS  (Res) 

9 

24  (11) 

26  (12) 

24  (10) 

27  (08) 

MPD  (Res) 

9 

03  (06) 

07  (08) 

13  (05) 

12  (05) 

PFB  (Res) 

9 

05  (08) 

10  (09) 

08  '(06) 

07  (06) 

Wr i tten 

9 

27  (12) 

29  (07) 

31  (06) 

28  (09) 

Ratings 

9 

02  (05) 

03  (05) 

05  (03) 

05  (04) 

HO-Total 

9 

33  (06) 

35  (06) 

29  (08) 

34  (06) 

JK-Total 

9 

46  (11) 

48  (09) 

46  (07) 

46  (08) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  stanaard  deviations.  Numbers  in  brackets  are  tfie  numbers  of 
predictor  scores  entering  prediction  equations.  Decimals  omitted. 

^  Number  of  M05  for  which  validities  were  computed. 

“  MOS-Appropr iate  Aptitude  Area  composite. 


differences  between  measures  with  respect  to  range  restriction  will  have  been 
removed . ) 

The  average  multiple  correlations  for  the  three  sets  of  ABLE  scores  are 
reported  in  Tables  7.11  and  7.12  (corrected  and  uncorrected,  respectively). 
The  multiple  correlations  for  the  second  set  of  alternate  ABLE  factor  scores 
(those  based  on  the  reduced  set  of  items)  were  consistently  higher  than  those 
for  the  other  two.  Note  that  the  validities  for  the  ABLE  rational  composites 
tended  to  be  the  lowest  of  the  three  sets  of  ABLE  scores. 


Incremental  Validities  for  the  Experimental  Battery  Predictors 
Over  the  ASVAB  Factors  (Based  on  Listwise  Deletion  Sample) 

Incremental  validity  results  for  the  Experimental  Battery  predictors 
over  the  ASVAB  factors  are  reported  in  Tables  7.13  and  7.14.  Table  7.13 
reports  the  average  multiple  correlations  of  each  set  of  Experimental  Battery- 
predictors  in  combination  with  the  four  ASVAB  factors.  These  results,  based 
on  the  listwise  deletion  sample,  were  adjusted  for  shrinkage  using  the 
Rozeboom  formula  and  corrected  for  range  restriction.  Numbers  that  are 
underlined  indicate  validities  higher  than  those  obtained  with  the  four  ASVAB 
factor  composites  alone  (which  are  reported  in  italics). 

The  results  indicate  that  the  spatial  composite  adds  slightly  to  the 
prediction  of  the  raw  and  residual  Core  Technical  and  General  Soldiering 
performance  factors,  as  well  as  to  the  written  iriethod  factor  and  tne  riands-on 
and  Job  Knowledge  total  scores.  They  also  show  that  the  seven  ABLE  composites 
contribute  substantially  to  the  prediction  of  the  raw  and  residual  rersonal 
Discipline  and  Physical  Fitness  performance  factors. 

Incremental  validity  results  which  have  not  been  corrected  for  range 
restriction  (but  wh'Ch  were  adjusted  for  shrinkage)  are  reported  in  Table 
7.14.  The  pattern  of  results  in  this  table  is  similar  to  those  reported  in 
Table  7.13;  however,  the  estimated  level  of  incremental  validity  is  actually 
higher  for  the  results  that  have  not  beer,  corrected  for  range  restriction  than 
for  those  for  which  the  correction  was  made.  The  estimates  are  generally 
three  points  higher  for  CTP,  GSP,  and  MPD,  but  about  six  points  higher  for  PFB 
as  incrementally  predicted  by  ABLE. 

The  adjusted  incremental  validity  results  for  each  of  the  three  sets  of 
ABLE  scores  over  the  ASVAB  factors  are  reported  in  Tables  7.15  and  7.16 
(corrected  and  uncorrected  for  range  restriction,  respectively).  All  three 
sets  of  scores  account  for  variance  in  the  raw  and  residual  Personal 
Discipline  and  Physical  Fitness  performance  factors  not  predicted  by  the  ASVAB 
factors.  This  incremental  validity  is  slightly  greater  (up  to  two  validity 
points)  for  the  factor  scores  based  on  the  reduced  number  (114)  of  ABLE  items. 


Table  7.11 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ABLE  Rational  Composites,  ABLE-168  Items,  and  ABLE-114  Items: 
Results  Corrected  for  Range  Restriction _ 


Criterion 

No.  of 
MOS^ 

ABLE 

Composites 

[7] 

ABLE- 

168 

[7] 

ABLE- 

114 

[7] 

CTP  (Raw) 

9 

.21  (.09) 

.25  (.07) 

.26  (.10) 

GSP  (Raw) 

8 

.23  (.14) 

.26  (.11) 

.28  (.13) 

ELS  (Raw) 

9 

.13  (.11) 

.15  (.12) 

.16  (.12) 

MPD  (Raw) 

9 

.14  (.11) 

.15  (.11) 

.17  (.12) 

PFB  (Raw) 

9 

.27  (.07) 

.27  (.07) 

.27  (.07) 

CTP  (Res) 

9 

.08  (.11) 

.12  (.09) 

.16  (.12) 

GSP  (Res) 

8 

.12  (.12) 

.14  (.14) 

.19  (.14) 

ELS  (Res) 

9 

.21  (.15) 

.21  (.15) 

.22  (.17) 

MPO  (Res) 

9 

.13  (.11) 

.14  (.12) 

.17  (.11) 

PPB  (Res) 

9 

.28  (.10) 

.25  (.10) 

.28  (.10) 

Written 

9 

.23  (.12) 

.24  (.11) 

.24  (.09) 

Ratings 

9 

.03  (.05) 

.03  (.05) 

.03  (.04) 

HO-Tota 1 

9 

.11  (.11) 

.13  (.12) 

.18  (.12) 

JK-Tota  1 

9 

.31  (.08) 

.32  (.08) 

.33  (.09) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of 
ored'Ctor  scores  entering  prediction  equations. 

Number  of  MOS  for  which  validities  were  computed. 


Table  7.12 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Listwise  Deletion 
Sample  for  ABLE  Rational  Composites,  ABLE-168  Items,  and  ABLE-114  Items: 
Results  Uncorrected  for  Range  Restriction _ 


Cr i ter  ion 

No .  of 

MOS" 

ABLE 

Composites 

[7] 

ABLE- 

168 

[7] 

able- 

114 

[7] 

CTP  (Raw) 

9 

.02  (.03) 

.03  (.06) 

.01  (.02) 

GSP  (Raw) 

8 

.05  (.08) 

.05  (.08) 

.09  (.071 

ELS  (Raw) 

q 

.04  (.06) 

.07  (.07) 

.07  (.08) 

MPD  (Raw) 

9 

.13  (.09) 

.14  (.09) 

.16  (.10) 

PFB  (Raw) 

9 

.27  (.07) 

.28  (.07) 

.27  (.07) 

CTP  (Res) 

9 

.00  (.00) 

.01  (.02) 

.01  (.02) 

GSP  (Res) 

8 

.03  (.06) 

.04  (.06) 

.05  (.07) 

ELS  (Res) 

9 

.07  (.091 

.08  (.09) 

.08  (.09) 

MPD  (Res) 

9 

.11  (.10) 

.13  (.10) 

.15  (.10) 

PFB  (Res) 

9 

.29  (.09) 

.29  (.10) 

.28  (.11) 

Written 

9 

.05  (.08) 

.04  (.08) 

.03  (.06) 

Ratings 

9 

.04  (.07) 

.04  (.06) 

.03  (.05) 

HO-Total 

9 

.01  (.02) 

.02  (.03) 

.03  (.05) 

JK-Tota 1 

9 

.08  (.10) 

.08  (.10) 

.09  (.10) 

Note:  Adjusted  for 

Shrinkage 

(Rozeboom  formula  8).  Numbers 

in  parentheses 

are  standard 

deviations 

Numbers  in 

brackets  are  the 

numbers  of 

predictor  scores  entering  prediction  equations. 
^  Number  of  MOS  for  which  validities  were  computed. 
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Table  7.13 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within-Job  for 
LVI  Listwise  Deletion  Sample  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 
AVOICE:  Results  Corrected  for  Range  Restriction _ 


ASVAB 


Factors 

A4  + 

A4  + 

A4  + 

A4-.- 

A4  + 

No.  of  (A4) 

Spatial 

Computer 

JOB 

ABLE  Comp. 

AVOICE 

Criterion  MOS^  [4] 

[5] 

[12j 

[7] 

[11] 

[12] 

CIP 

( R  aw ) 

Q 

6P 

(13) 

63 

(13) 

61 

(14) 

61 

(13) 

61 

(13) 

62 

(13) 

GSP 

(Raw) 

8 

66 

(07) 

68 

(07) 

66 

(07) 

66 

(07) 

66 

(07) 

66 

(07) 

ELS 

(Raw) 

9 

37 

(12) 

36 

(13) 

35 

(13) 

36 

(13) 

34 

(17) 

33 

(16) 

MPO 

(Raw) 

9 

17 

(13) 

16 

(14) 

16 

(15; 

14 

(15) 

23 

(14) 

10 

(15) 

PFB 

(Raw) 

9 

16 

(06) 

13 

108) 

09 

(08) 

17 

(08) 

.30 

(06) 

12 

(10) 

CTP 

( Res ) 

9 

46 

(17) 

47 

(17) 

44 

(18) 

45 

(18) 

43 

(19) 

46 

(19) 

GSP 

(Res) 

8 

51 

(10) 

53 

(09) 

51 

(10) 

50 

(10) 

50 

(10) 

50 

(10) 

ELS 

(Res) 

9 

46 

(18) 

47 

(IS) 

44 

(21) 

45 

(21) 

45 

(22) 

44 

(21) 

MPC 

(Res) 

0 

18 

(13) 

15 

(14) 

15 

(14) 

)4 

(14) 

22 

(14) 

12 

(13) 

PFB 

(Res'; 

9 

20 

(10) 

18 

(12) 

i  .1 

(11) 

20 

(in 

34 

(10) 

18 

(13) 

Written 

9 

54 

(13' 

55 

(13) 

51 

(18) 

54 

(13) 

54 

(12) 

52 

(17) 

Rat 

i  I'l  g  5 

9 

12 

(09) 

i  i 

(08) 

09 

(10) 

09 

(10) 

09 

(OS) 

05 

(08) 

HO-' 

Iota  1 

9 

50 

(14) 

52 

(13) 

49 

(14) 

49 

(15) 

48 

(14) 

49 

(15) 

JK-' 

Total 

9 

71 

(08) 

72 

(08) 

71 

(09) 

71 

(08) 

71 

(08) 

71 

(08) 

Note:  Ac'jjSte'J  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numoers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Multiple  Rs  for  ASVAB 
Factors  alone  are  in  italics.  Underliiied  numbers  denote  multiple  Rs 
greater  than  for  ASVAB  Factors  alone.  Decimals  emitted. 

°  Number  of  MOS  tor  which  validities  were  computed. 


•  7  1 


Table  7.14 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within-Job  fnr 
l.VI  Listwise  Deletion  Sample  for  Spatial,  Computer,  JOB,  ABLE  Composites,  and 
AVOICE:  Results  Uncorrected  for  Range  Restriction _ 


Cr  i 

terion 

No.  of 
MGS'* 

ASVAB 

Factors 

(A4) 

[4] 

A4  + 

Spatial 

[51 

A4+ 

Computer 

[12] 

A4  + 

JOB 

[7] 

A4  + 

ABLE  Comp. 
[11] 

A4  + 

AVOICE 

[12] 

CTP 

(Raw) 

.9 

41  (09) 

42 

(10) 

38 

(11) 

40 

(08) 

38 

(10) 

40 

(08) 

GSP 

(Raw) 

8 

46  (06) 

50 

(05) 

45 

(09) 

45 

(06) 

44 

(07) 

44 

(08) 

ELS 

(Raw) 

9 

19  (08) 

18 

(08) 

19 

(09) 

19 

(09) 

18 

(10) 

16 

(10' 

MPO 

( Raw ) 

9 

09  (08) 

08 

(07) 

07 

(07) 

06 

(07) 

17 

(09) 

04 

(06) 

PFG 

(  R  dW  ) 

9 

08  (09) 

07 

(09) 

06 

(08) 

n 

(09) 

28 

(06) 

08 

(10) 

CTP 

(Res) 

9 

30  (09) 

31 

(10) 

27 

(11) 

29 

(10) 

25 

(12) 

29 

(12) 

GSP 

(Res) 

8 

35  (04) 

39 

(03) 

34 

(07) 

35 

(05) 

33 

(06) 

34 

(06) 

ELS 

(Res) 

9 

26  (12) 

26 

(11) 

25 

(11) 

25 

(11) 

26 

(12) 

24 

(10) 

MPO 

(Res) 

9 

07  (08) 

06 

(07) 

07 

(07) 

04 

(06) 

14 

(11) 

03 

(04) 

PFB 

(Res) 

9 

10  (09) 

09 

(09) 

07 

(09) 

13 

(09) 

30 

(09) 

09 

(11) 

Wr i tten 

9 

29  (07) 

31 

(08) 

25 

(11) 

28 

(07) 

27 

(10) 

25 

(11) 

Rat 

i  ngs 

9 

03  (05) 

02 

(04) 

02 

(03) 

03 

(04) 

(06) 

00 

(01) 

HO- 

Totd  1 

9 

35  (06) 

37 

(06) 

33 

(08) 

34 

(06) 

31 

(08) 

33 

(08) 

JK- 

Tota  1 

9 

48  (09) 

(10) 

45 

(16) 

48 

(OS) 

48 

(08) 

46 

(10) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Multiple  Rs  for  ASVAB 
Factors  alone  are  in  italics.  Underlined  numbers  denote  multiple  Rs 
greater  than  for  ASVAB  Factors  alone.  Decimals  omitted. 

Number  of  MOS  for  which  validities  were  computed. 


Table  7.15 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within-Job  for 
LVI  Lis^wise  Deletion  Sample  for  ABLE  Rational  Composites,  ABLE-168  Items,  and 
ABLE-114  Items:  Results  Corrected  for  Range  Restriction _ 


ASVAB 

A4  + 

A4  + 

A4  + 

Factors 

ABLE 

ABLE- 

ABLE- 

No.  of 

(A4) 

Comp. 

168 

114 

Cr  iter  ion 

MGS'* 

[A] 

[11] 

[11] 

[11] 

CTP  (Raw) 

9 

62  (13) 

61  (13) 

61  (13) 

61  (13) 

GSP  (Raw) 

8 

66  (07) 

66  (07) 

66  (07) 

o6  (07) 

SL5  (Raw) 

9 

37  (12) 

3*’  (17) 

35  (17) 

36  (15) 

MPD  (Raw) 

9 

17  (13) 

23  (14) 

24  (la) 

25  (14) 

?FB  (Raw) 

9 

16  (06) 

30  (06) 

31  (06) 

31  (06) 

CTP  (Res) 

9 

46  (17) 

43  (19) 

44  (20) 

44  (19) 

GSP  (Res) 

8 

51  (10) 

46  (18) 

50  (10) 

50  (10) 

51  (10) 

ELS  (Res) 

9 

45  (22) 

45  (22) 

45  (22) 

MPD  (Res) 

9 

18  ^13) 

22  (14) 

23  (14) 

24  (14) 

PFE  (Res) 

9 

20  (10) 

34  (10) 

34  (10) 

34  (10) 

Wr  i  tten 

9 

54  (13) 

54  (12) 

54  (11) 

53  (12) 

Ratings 

9 

19  {r\Q\ 

*  \  ^  j 

00  /OQ'. 

V  ✓  \  / 

09  (09) 

09  (09) 

HO-Tota  i 

9 

50  (14) 

48  (14) 

48  (14) 

48  (15) 

JK-Tota  1 

9 

71  (08) 

71  (08) 

71  (08) 

71  (08) 

Note:  Adjusted  for  shrinkage  (Rozeboom  fcmula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Multiple  Rs  for  ASVAB 
Factors  alone  are  in  italics.  Underlined  numbers  denote  multiple  Rs 
greater  than  for  ASVAB  Factors  alone.  Decimals  omitted. 

^  Number  of  M05  for  which  validities  'were  computed. 


Table  7.16 


Mean  of  Incrementdl  Correlations  Over  ASVAB  Factors  Computed  Within-Job  for 
LVI  Listwise  Deletion  Sample  for  ABLE  Rational  Composites,  ABLE-168  Items,  and 
ABLE-114  Items;  Results  Uncorrected  for  Range  Restriction _ 


Cr i ter  ion 

No.  of 
MOS* 

ASVAB 

Factors 

(A4) 

[4] 

A4  + 

ABLE 

Comp. 

[11] 

A4+ 

ABLE- 

168 

[11] 

A4  + 

ABLE- 

114 

[11] 

C' ^  (Raw) 

9 

41  (09) 

38  no) 

38  (10) 

38  (10) 

GSP  (Raw) 

8 

46  (06) 

44  (07) 

45  (07) 

46  (06) 

ELS  (Raw) 

9 

19  (08) 

18  (10) 

20  (10) 

20  (10) 

MPD  (Raw) 

9 

C9  (08) 

17  (09) 

18  (10) 

19  (09) 

PFB  (Raw) 

9 

08  (09) 

28  (06) 

28  (06) 

21  (06) 

CTP  (Res) 

9 

30  (09) 

25  (12) 

25  (12) 

25  (12) 

GSP  (Res) 

8 

36  (04) 

33  (06) 

34  (06) 

35  (05) 

ELS  (Res) 

9 

26  (12) 

26  (12) 

27  (12) 

27  (12) 

MPD  (Resl 

9 

07  (08) 

14  (11) 

16  (10) 

18  (09) 

PFB  (Res) 

Q 

10  (09) 

30  (09) 

30  (09) 

30  (10) 

V/r  i  tten 

9 

29  (07) 

27  (10) 

27  (09) 

26  (09) 

Ratings 

9 

03  (05) 

M  (06) 

05  (06) 

M  (07) 

HO-Tota 1 

9 

35  (06) 

31  (08) 

32  (08) 

32  (07) 

JK-Tota 1 

9 

48  (09) 

48  (08) 

48  (07) 

48  (07) 

Note:  Adjusted  for  shrinkage  (Rozeboom  formula  8).  Numbers  in  parentheses 
are  standard  deviations.  Numbers  in  brackets  are  the  numbers  of 
predictor  scores  entering  prediction  equations.  Multiple  Rs  for  ASVAB 
factors  alone  are  in  italics.  Underlined  numbers  denote  multiple  Rs 
greater  than  for  ASVAB  Factors  alone.  Decimals  omitted. 

^  .Number  of  MOS  for  which  validities  were  computed. 
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Multiple  Correlations  and  Incremental  Validities  Over 
the  ASVAB  Factors  for  the  Experimental  Battery 
Predictors  (Based  on  the  Setwise  Deletion  Samples) 

Multiple  correlations  for  the  spatial  composite,  the  eight  computer 
composites,  the  three  JOB  composites,  the  seven  ABLE  composites,  and  the  eight 
AVOICE  composites  based  on  the  setwise  deletion  samples  described  above  are 
reported  in  Table  7.17.  Like  the  validity  results  based  on  the  listwise 
deletion  sample  reported  in  Table  7.7,  these  results  have  been  adjusted  for 
shrinkage  using  the  Rozeboom  formula  and  corrected  for  range  restriction. 

The  multiple  correlations  computed  with  the  setwise  samples  are  very 
similar  to  those  computed  -.vith  the  listwise  sample.  However,  there  is  a 
consistent  difference  between  the  two  sets  of  results;  specifically,  the 
multiple  correlations  based  on  the  setwise  samples  are  generally  one  to  three 
val idity  points  higher. 

As  noted  earlier,  we  did  not  expect  the  validities  to  either  increase  or 
decrease  systematically  across  the  listwise  and  setwise  deletion  samples. 
Furthermore,  we  can  offer  no  plausible  theoretical  or  statistical  explanation 
for  these  differences.  Therefore,  attempting  to  interpret  these  findings  may 
not  be  appropriate. 

Incremental  validity  results  associatea  with  the  setwise  deletion 
samples  can  be  found  in  Tables  7.18  and  7.19.  Table  7.18  reports  the  multiple 
correlations  for  the  four  ASVAB  factors  alone  (as  computed  separately  in  each 
of  the  setwise  deletion  samples),  whereas  Table  7.19  reports  the  multiple 
correlations  for  the  four  ASVAB  factors  along  with  each  set  of  predictors  in 
the  Experimental  Battery.  Numbers  unde:*liried  in  Table  7.19  indicate  multiple 
correlations  that  are  higher  than  those  based  on  ASVAB  alone.  Once  again, 
results  are  adjusted  for  shrinkage  using  the  Rozeboom  formula  and  corrected 
for  range  restriction. 

The  incremental  idity  results  based  on  the  setwise  samples  are 
practically  identical  those  based  on  the  listwise  sample.  Again,  the 
primary  difference  between  the  two  sets  of  results  is  that  the  level  of 
validities  are  sometimes  one  or  two  points  lower  for  the  listwise  sample  than 
for  the  setwise  samples. 


COMPARISON  BETWEEN  VALIDITY  RESULTS  OBTAINED  WITH 
LONGITUDINAL  AND  CONCURRENT  SAMPLES 

The  final  set  of  results  concern  the  comparison  between  the  validity 
results  associated  with  the  longitudinal  data  (i.e.,  LVI)  and  those  reported 
for  the  concurrent  validation  data  (CVI).  Table  7.20  reports  the  multiple 
correlations  for  the  ASVAB  factors  and  each  set  of  exper imepita  1  predictors  as 
computed  for  tne  listwise  sample  in  both  data  sets.  Note  that  there  are 
differences  between  the  CVI  and  LVI  data  in  the  number  of  predictor  composites 
included  in  some  of  the  experimental  predictor  sets.  In  particular,  for  the 
CVI  analyses  there  were  only  six  computer  composites,  four  ABLE  composites, 
and  six  AVOICE  composites. 


Table  7.17 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  LVI  Setwise  Deletion 
Samples  for  Spatial,  Computer,  JQD,  ABLE  Composites,  and  AVOICE _ 


Criterion 

No.  of 
MOS* 

Spatial 

[1] 

Computer 

[3] 

JOB 

f3l 

ABLE 

Composites 

f7] 

AVOICE 

[8] 

CTP  (Raw) 

9 

58  (11) 

49 

(16) 

31  (13) 

21  (09) 

39  (07) 

GSP  (Raw) 

8 

65  (06) 

55 

(08) 

32  (13) 

24  (14) 

38  (07) 

ELS  (Raw) 

9 

33  (08) 

30 

(15) 

19  (14) 

12  (11) 

20  (12) 

MPO  (Raw) 

9 

14  (11) 

10 

(16) 

06  (13) 

15  (11) 

05  (11) 

PFB  (Raw) 

9 

08  (04) 

13 

(07) 

07  (06) 

28  (07) 

09  (09) 

CTP  (Res) 

9 

43  (15) 

31 

(22) 

17  (12) 

10  (11) 

29  (09) 

GSP  (Res) 

3 

51  (08) 

40 

(10) 

21  (11) 

14  (12) 

28  (09) 

ELS  (Res) 

9 

41  (13) 

36 

(20) 

24  (-3) 

21  (15) 

26  (06) 

MPD  (Res) 

9 

13  (12) 

10 

(16) 

06  (11) 

15  (11) 

07  (13) 

PFB  (Res) 

9 

11  (08) 

10 

(11) 

09  (06) 

30  (10) 

12  (10) 

Wr i tten 

9 

51  (11) 

46 

(16) 

31  (17) 

25  (11) 

32  (15) 

fiat  ings 

9 

u9  (08) 

09 

(09) 

07  (08) 

04  (06) 

03  (07) 

HO-Total 

9 

50  (11) 

38 

(15) 

20  (13) 

13  (11) 

30  (07) 

JK-Tota  1 

9 

66  (07) 

60 

(10) 

38  04) 

30  (08) 

42  (08) 

Note:  Results  corrected  for  range  restriction  and  adjusted  for  shrinkage 

(Rozeboom  formula  8).  N'j'ibers  in  parentheses  are  standard  deviations. 
Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering 
prediction  equations.  Decimals  omitted. 

^  Number  of  MOS  for  which  validities  were .computed. 


Table  7.18 


Mean  of  Multiple  Correlations  Computed  Within-Job  for  ASVAB  Factors  Within 
Each  of  the  Five  LVI  Setwise  Deletion  Samples _ 


ASVAB 

ASVAB 

ASVAB 

ASVAB 

ASVAB 

Factor 

Factor 

Factor 

Factor 

Factor 

No.  of  (Spatial) 

(Computer) 

(JOB) 

(ABLE  Comp. ) 

(AVOICE) 

Criterion  MOS^  [4] 

[4] 

[4] 

[4] 

[4] 

CTP 

(Raw) 

9 

63 

(10) 

62 

(11) 

63 

(11) 

62 

(12) 

64 

(11) 

GSP 

(Raw) 

8 

66 

(07) 

65 

(07) 

67 

(07) 

66 

(07) 

67 

(07) 

ELS 

(Raw) 

9 

37 

(10) 

37 

(12) 

37 

(11) 

36 

(11) 

37 

(11) 

MPD 

(Raw) 

9 

16 

(13) 

15 

(13) 

15 

(12) 

16 

(13) 

16 

(12) 

PFB 

(Raw) 

9 

16 

(08) 

19 

(05) 

16 

(07) 

15 

(09) 

16 

(09) 

CTP 

(Res) 

9 

47 

(12) 

46 

(13) 

47 

(14) 

47 

(14) 

48 

(13) 

GSP 

(Res) 

8 

51 

(06) 

50 

(08) 

51 

(08) 

51 

(08) 

52 

(07) 

ELS 

(Res) 

9 

47 

(12) 

46 

.15) 

47 

(14) 

46 

(14) 

47 

(14) 

MPD 

(Res ) 

9 

15 

(13) 

14 

(12) 

14 

(13) 

14 

(14) 

16 

(13) 

PFB 

(Res) 

9 

21 

(iO) 

21 

(09) 

20 

(09) 

20 

(11) 

21 

(10) 

Wr  i ' 

tten 

9 

56 

(13) 

55 

(12) 

58 

(H) 

55 

(14) 

56 

(14) 

Ratings 

9 

10 

(10) 

11 

(11) 

11 

(08) 

11 

(10) 

10 

(10) 

HO-' 

Tota  1 

9 

51 

(09) 

50 

(li) 

50 

(12) 

50 

(11) 

51 

(10) 

JK- 

rota  1 

9 

71 

(09) 

71 

(08) 

72 

(08) 

71 

(09) 

72 

(09) 

Note:  Results  corrected  for  range  restriction  and  adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  parentheses  are  standard  deviations. 
Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering 
preaiction  equations.  Decimals  omitted. 


Number  of  MOS  for  which  validities  were  computed. 


Table  7.19 


Mean  of  Incremental  Correlations  Over  ASVAB  Factors  Computed  Within-Job  for 
LVI  Setwise  Deletion  Samples  for  Spatial,  Computer,  JOS,  ABLE  Composites,  and 


AVOICE 


Criterion 

No.  of 
MOS^ 

ASVAB 

Factors  (A4) 

+  Spatial 

_ liT _ 

A4  + 

Computer 

ri2] 

A4  + 

JOB 

[7] 

A4+  ABLE 
Composites 

_  til] 

A4  + 

AVOICE 

[12] 

CTP  (Raw) 

9 

64 

(10) 

61  (11) 

63  (11) 

61  (12) 

64  (11) 

GSP  (Raw) 

8 

69 

(06) 

66  (07) 

67  (07) 

66  (08) 

66  (07) 

ELS  (Raw) 

9 

37 

(10) 

36  (14) 

37  (11) 

36  (13) 

36  (11) 

MPD  (Raw) 

9 

15 

(13) 

15  (15) 

12  (13) 

24  (13) 

11  (14) 

PFB  (Raw) 

9 

15 

(08) 

17  (05) 

IZ  (07) 

32  (04) 

15  (10) 

CTP  (Res) 

9 

48 

(12) 

45  (14) 

46  (14) 

45  (14) 

47  (14) 

GSP  (Res) 

8 

54 

(06) 

50  (08) 

51  (08) 

50  (07) 

50  (07) 

ELS  (Res) 

9 

47 

(12) 

43  (20) 

46  (15) 

46  (15) 

46  (14) 

MPD  (Res) 

9 

14 

(13) 

13  (15) 

13  (13) 

22  (12) 

11  (14) 

PFB  (Re?) 

9 

20 

(11) 

18  (11) 

20  (10) 

36  (08) 

21  (11) 

Written 

9 

57 

(13) 

53  (17) 

58  (12) 

55  (13) 

54  (18) 

Ratings 

9 

10 

(09) 

11  (11) 

11  (09) 

n  (07) 

06  (09) 

HO-Tota 1 

9 

53 

(09) 

49  (11) 

50  (12) 

49  (11) 

50  (11) 

JK-Tota  1 

9 

21 

(08) 

71  (09) 

72  (08) 

71  (09) 

71  (09) 

Note:  Results  corrected  for  range  restriction  and  adjusted  for  shrinkage 

(Rozeboom  formula  8).  Numbers  in  parentheses  are  standard  deviations. 
Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering 
prediction  equations.  Underlined  numbers  denote  multiple  Rs  greater 
than  for  ASVAB  Factors  alone  (as  reported  in  Table  7.18).  Decimals 
omitted . 

^  Number  of  MOS  for  which  validities  were  computed. 


Table  7.20 


Comparison  of  Mean  Multiple  Correlations  Computed  Within-Job  for  LVI  and  CVI 
Listwise  Deletion  Samples  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE 
Composites,  and  AVOICE:  Results  Corrected  for  Range  Restriction  and  Adjusted 
for  Shrinkage  (Claudy  Formula) _ 


ASVAB  ABLE 

Factors  Spatial  Computer  JOB  Comp.  AVOICE 

No.  of  LV  CV  LV  CV  LV  CV  LV  CV  LV  CV  LV  CV 


Criterion 

MOS^ 

[4] 

[4] 

[1] 

[1] 

[8] 

[6] 

[3]  [3] 

[7]  [4] 

[8] 

[6] 

CTP  (Raw) 

9 

63 

63 

57 

56 

50 

53 

31 

29 

27 

26 

41 

35 

GSP  (Raw) 

8 

67 

65 

64 

63 

57 

57 

32 

30 

29 

25 

40 

34 

ELS  (Raw) 

9 

39 

31 

32 

25 

34 

26 

22 

19 

20 

33 

25 

24 

MPD  (Raw) 

9 

22 

16 

14 

12 

15 

12 

11 

11 

22 

32 

11 

13 

PFB  (Raw) 

c 

21 

20 

10 

10 

17 

11 

12 

11 

31 

37 

15 

12 

CTP  (Res) 

9 

48 

47 

42 

37 

35 

37 

20 

21 

18 

22 

33 

28 

GSP  (Res) 

8 

53 

49 

51 

48 

44 

41 

22 

22 

19 

21 

31 

26 

ELS  (Res) 

9 

48 

46 

41 

41 

40 

38 

25 

27 

26 

31 

29 

32 

MPD  (Res) 

9 

23 

19 

14 

15 

14 

13 

12 

10 

21 

28 

13 

15 

PFB  (Res) 

9 

24 

21 

12 

11 

17 

14 

11 

10 

32 

35 

16 

14 

Written 

9 

56 

62 

49 

55 

47 

54 

31 

28 

29 

21 

33 

32 

Ratings 

9 

16 

15 

09 

07 

17 

08 

10 

08 

09 

18 

09 

09 

Note:  Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering 
prediction  equations.  Decimals  omitted. 

^  Number  of  MGS  for  which  validities  were  computed. 


The  results  in  Table  7.20  have  been  adjusted  for  shrinkage  and  corrected 
for  range  restriction.  As  previously  indicated,  the  adjustments  for  shrinkage  in 
this  step  were  all  made  using  the  Claudy  formula,  rather  than  the  Rozeboom.  This 
is  because  the  CVI  results  have  not  been  reported  using  the  Rozeboom  formula. 

The  primary  difference  between  the  two  corrections  is  that  the  Claudy  formula 
estimates  the  multiple  correlation  of  the  population  regression  equation  when 
applied  to  the  population,  whereas  the  Rozeboom  estimates  the  multiple 
correlation  of  the  sample-based  regression  equation  when  applied  to  the 
population. 

In  that  the  population  regression  equation  cannot  actually  be  obtained  in 
an  applied  prediction  situation,  we  believe  that  it  is  generally  more  appropriate 
to  use  the  Rozeboom  correction  for  estimaiing  the  level  of  validity  of  a  set  of 
predictors.  However,  because  the  relative  sizes  of  the  validity  coefficients 
across  the  different  predictor  sets  and  criterion  constructs  should  be  unaffected 
by  the  particular  adjustment  formula  used,  the  comparison  between  the  LVI  and 
CVI  results  based  on  the  Claudy  adjustment  should  be  approximately  the  same  as  a 
comparison  between  these  two  sets  of  results  based  on  the  Rozeboom  adjustment. 
Indeed,  a  comparison  of  the  Claudy-  and  Rozeboom-ad justed  results  for  the  LVI 
sample  shows  that  the  pattern  of  results  is  almost  identical  (although,  as  would 
be  expected,  the  level  of  validities  is  higher  for  the  C laudy-ad justed  results). 

Overall  Results 

The  results  in  Table  7.20  demonstrate  that  the  pattei^ns  and  levels  of 
validities  are  very  similar  across  the  two  sets  of  analyses.  Still,  there  are 
several  differences  worth  pointing  out.  Specifically,  in  comparison  to  the 
results  of  the  CVI  analyses:  (a)  the  LVI  validities  of  the  "cognitive"  predictors 
(i.e.,  ASVAB,  spatial,  computer)  for  predicting  the  "will  do"  performance  factors 
(ELS,  MPO,  and  PEB)  are  higher;  (b)  the  LVI  validities  of  the  ABLE  composites  for 
predicting  the  "will  do"  performance  factors  are  lower;  and  (c)  the  LVI 
validities  of  the  AVOICE  composites  for  predicting  the  "can  do"  performance 
factors  (CTP  and  GSP)  are  higher.  CVI  results  were  not  reported  for  the  Hands-On 
and  Job  Knowledge  total  scores. 

Finally,  the  incremental  validity  results  for  the  CVI  and  LVI  analyses  for 
the  five  raw  performance  factors  (CVI  resu’ts  were  not  reported  f\:r  the  other 
criteria)  are  reported  in  Table  7.21.  The.  results  are  adjusted  'cr  shrinkage 
(again  using  Claudy)  and  corrected  for  range  restriction.  Once  more.,  the  results 
are  very  similar  across  the  two  sets  of  analyses.  However,  the  LVI  ’ncremental 
validities  associated  with  the  ABLE  were  not  quite  as  large  as  those  '"eported  in 
the  CVI  analyses. 


Further  Exploration  of  ELS  and  ABLE 

As  shown  in  the  data  reported  above,  the  largest  difference  between  the  CVI 
and  LVI  validation  results  was  in  the  prediction  of  the  Effort  and  Leadership 
(ELS)  performance  factor  with  the  ABLE  basic  scores.  Corrected  for  restriction 
of  range  and  for  shrinkage,  the  validity  of  the  four  ABLE  composite  scores  in  CVI 
was  .33  for  ELS  and  the  /alidity  of  the  seven  ABLE  factor  scores  in  LVI  was  .20. 
When  cast  against  the  variability  in  results  across  studies  in  the  extant 
literature,  such  a  difference  may  not  seem  all  that  large  or  very  unusual. 
Howeve-',  since  the  obtained  results  from  CVI,  CVII,  and  LVI  have  been  so 
consistent,  in  terms  of  the  expected  convergent  and  divergent  results,  we 


Table  7.21 


Comparison  of  Mean  Incremental  Correlations  Computed  Within- Job  for  LVl  and  CVI 
Listwise  Deletion  Samples  for  ASVAB  Factors,  Spatial,  Computer,  JOB,  ABLE 
Composites,  and  AVOICE:  Results  Corrected  for  Range  Restriction  and  Adjusted  for 


ASVAB 

Factors 

Spatial 

Computer 

JOB 

ABLE 

Composites 

AVOICE 

Criterion 

No.  of 
MOS' 

LV  CV 

[4]  [4] 

LV 

[5] 

CV 

[5] 

LV  CV 
[12]  [10] 

LV  CV 
[7]  [7] 

LV 

[HI 

CV 

[8] 

LV 

[121 

CV 

[10] 

CTP 

(Raw) 

9 

63  63 

64 

65 

63 

64 

63 

63 

63 

63 

64 

64 

GSP 

(Raw) 

8 

67  65 

69 

68 

68 

67 

0/ 

66 

68 

66 

68 

66 

ELS 

(Raw) 

9 

39  31 

39 

32 

41 

32 

40 

33 

41 

42 

40 

35 

MPD 

(Raw) 

9 

22  16 

22 

17 

25 

17 

22 

19 

31 

35 

24 

19 

PFB 

(Raw) 

9 

21  20 

20 

22 

23 

22 

24 

22 

33 

41 

24 

24 

Note:  Numbers  in  brackets  are  the  numbers  of  predictor  scores  entering 

prediction  equations.  Multiple  Rs  for  ASVAB  Factors  alone  are  in  italics; 
underlined  numbers  denote  multiple  fts  greater  than  for  ASVAB  Factors 
alone.  Decimals  omitted, 

^  Number  of  MOS  for  which  validities  were  computed. 


subjected  this  particular  difference  to  a  series  of  additional  analyses  in  an 
attempt  to  determine  the  source  of  the  discrepancy. 


Potential  Causes 

First,  the  discrepancy  does  not  seem  to  arise  from  any  general 
deterioration  in  the  measurement  properties  of  either  the  ABLE  or  the  ELS 
composite  in  the  LVI  sample.  For  example,  while  the  correlacion  of  the  ABLE 
v/ith  ELS  and  MPD  went  down,  the  ABLE's  correlations  with  CTP  ana  GSP  went  ud 
slightly.  Similarly,  a  decrease  in  the  validity  with  wmch  ELS  is  predicted 
is  characteristic  only  of  the  ABLE.  The  validities  of  the  cognitive  measures, 
the  JOB,  and  AVOICE  for  predicting  ELS  actually  increased  by  varying  amounts. 
Consequently,  the  decrease  in  validity  seems  to  be  specific  to  the  ABLE/ELS 
correlation  and,  to  a  lesser  extent,  the  ABLE/MPO  correlation. 

Other  potential  sources  of  the  discrepancy  that  might  exert  more 
specific  effects  are  the  following: 

Differences  in  the  way  the  ABLE  was  scored  in  CVI  vs. 

LVI.  The  CVJ  able  composite  was  composed  of  four 
rationally  defined  construct  scores  while  the  seven 
LVI  scores  were  defined  by  the  results  of  a  factor 
analysis  (see  Chapter  2). 
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A  possible  response  bias  in  the  LVI  ABLE  that  affects 
differentially  the  validities  for  predicting  different 
components  of  performance. 

Different  content  for  Effort/Leadership  in  CVI  versus 
LVI.  For  example,  the  rating  scales  for  expected 
combat  performance  were  a  part  of  ELS  for  CVI  but  not 
for  LVI. 

Possible  differences  in  the  construct  being  measured 
by  ELS  in  CVI  versus  LVI.  That  is,  because  of  the 
rater/ratee  cohort  differences,  the  ratings  may 
actually  have  somewhat  different  determinants. 

In  an  effort  to  learn  more  about  these  potential  sources  of  the 
discrepancy,  a  number  of  additional  analyses  were  carried  out.  While 
obtaining  a  definitive  answer  would  require  additional  experimentation  and 
data  collection,  the  available  data  base  does  provide  a  great  deal  of  useful 
information,  the  most  relevant  aspects  of  which  are  summarized  below. 

Results  of  Followup  Analysis 

fhe  followiip  analyses  were  able  to  rule  out  two  possible  additional 
sources  of  the  CVI/LVI  validity  differences.  First,  differences  in  the 
composition  and  number  of  ABLE  basic  scores  from  CVI  to  LVI  do  not  account  for 
the  differences  in  patterns  of  validity.  We  recomputed  ABLE  composites  for 
the  LVI  data  using  the  composite  scoring  rules  from  CVI  to  determine  whether 
differences  in  CVI  to  LVI  validities  were  related  to  the  composition  and 
number  of  ABLE  composites.  As  shown  in  Table  7.22,  the  principal  differences 
between  LVI  validities  for  the  ABLE  scored  using  CVI  keys  versus  scored  using 
LVI  keys  were:  (a)  validities  against  CTP  and  GSP  dropped  somewhat  when  the 
CVI  key  was  used,  but  (b)  there  were  no  differences  between  CVI  and  LVI  keyed 
validities  for  the  "will  do"  criteria  (i.e.,  validities  did  not  go  up  when  the 
CV  key  was  used  with  LVI  data). 

Second,  differences  in  the  composition  of  the  Effort/Leadership  factor 
score  from  CVI  to  LVI  do  not  account  for  differences  in  validity.  The  LVI-ELS 
criterion  has  fewer  ratings  than  the  CVl-ELS  criterion  did,  making  the 
weignting  of  Ratings  to  Awards  smaller  than  it  was  in  CVI.  We  reweighted  the 
Rating  and  Awards  components  of  ELS  to  make  their  relative  contribution  to  the 
construct  more  similar  to  CVI.  We  then  compared  the  validities  resulting  from 
both  ELS  scores.  As  shown  in  Table  7.22,  there  was  essentially  no  difference 
between  them. 

The  available  evidence  does  not  rule  out  two  other  possible  explanations 
for  the  different  ABLE/ELS  correlations. 

First,  there  may  have  been  a  change  in  the  nature  of  the  construct  being 
measured  by  the  ELS  criterion  components,  which  may  account  for  the  lower 
ABLE/ELS  validity  in  the  LVI  sample.  That  is,  the  true  score  variance  of  the 
determinants  of  ELS  might  be  different  for  CVI  and  LVI;  for  example,  the 
sample  shows  greater  variabiHty  in  skill  but  more  uniform  levels  of 
motivational  determinants. 
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Table  7.22 


Multiple  Correlations,  Averaged  Over  MOS,  for  Alternative  Sets  of  ABLE  Scores 
With  Selected  Criterion  Scores  in  the  LVI  Sample _ 


Criterion 

ABLE 

Rat iona  1 
Compos i tes 
(7) 

A6LE- 

168 

ABLE- 

114 

ABLE  Compos 
LV 

ites  (4) 

CV 

CTP  (Raw) 

.27 

.30 

.31 

.24 

.26 

GSP  (Raw) 

.29 

.31 

.33 

.24 

.25 

ELS  (Raw) 

.20 

.22 

.24 

.20 

.33 

EL2"  (Raw) 

.19 

.21 

.23 

.20 

-- 

MPO  (Raw) 

.22 

.22 

.24 

.21 

.32 

PFB  (Raw) 

.31 

.32 

.31 

.31 

.37 

CTP  (Res) 

.18 

.21 

.23 

.17 

.22 

GSP  (Res) 

.19 

.22 

.25 

.17 

.21 

ELS  (Res) 

.26 

.27 

.29 

.23 

.31 

EL2"  (Res) 

.26 

.27 

.29 

.22 

-- 

MPD  (Res) 

.21 

.22 

.23 

.20 

.28 

PFB  (Res) 

.32 

.33 

.33 

.31 

.25 

Wr 1 tten 

.29 

.29 

.29 

.23 

.21 

Rat i ngs 

.09 

.11 

.12 

.10 

.18 

Note.  Corrected  for  restriction  of  range  and  adjusted  for  shrinkages. 
^  £L2  =  LV  recalculated  on  CV  basis. 


To  address  this  issue,  we  compared  the  CVI  versus  LVI  intercorrelations 
among  the  variables  constituting  the  ELS  criterion.  The  zero-oroer 
correlations  between  the  components  of  ELS  and  the  Hands-On,  Job  Knowledge, 
and  ASVAB  composite  scores  were  also  compared.  As  shown  in  Table  7.23.  the 
LVI -ELS  supervisor  ratings  correlate  more  highly  with  the  three  "non-Speed” 
ASVAB  factor  scores  than  did  CVI-ELS  supervisor  ratings,  suggesting  a  greater 
reliance  on  ability  and  skill  determinants. 

Second,  there  is  evidence  for  the  possible  effects  of  some  degree  of 
response  bias  in  the  ABLE  during  the  LVI  data  collection.  Recall  that  the 
Experimental  Battery  was  administered  at  the  beginning  of  Basic  Training  just 
a  few  days  after  induction.  The  new  recruits  may  easily  have  ascribed 
operational  importance  to  the  scores  even  though  they  were  informed  otherwise. 

As  shown  in  Table  7.24,  the  Social  Desirability  scale  scores  are  almost 
one-half  standard  deviation  higher  for  the  LVI  sample  than  for  the  CV!  sample. 
Also,  mean  scores  on  some  of  the  individual  ABLE  content  scales  are  higher  for 
tne  LVI  sample  than  for  the  CV!  sample.  Internal  Control,  Traditional  '/dlues, 
Nondelinquency,  Locus  of  Control,  and  Dependability  yielo  the  greatest  CVI  to 
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Table  7.23 


Intercorrelations  of  the  Four  ASVAB  Factors  and  the  Basic  Criterion  Scores 
Within  Effort/Leadership  (ELS),  Computed  for  Both  CVl  and  LVl  Samples _ 


Awards 

Overall 

Effect. 

(P) 

ELS 

Rating 

Comp. 

(P) 

Overa  1 1 
Effect. 
(S) 

ELS 

Rating 
Comp . 

(S) 

CVl  Sample 

ELS  Components 

Awards 

1.00 

.  14 

.17 

.13 

.15 

Overall  Effectiveness 

( Peer) 

.14 

1.00 

.79 

.38 

.40 

ELS  Rating  Composite 

(Peer) 

.17 

.79 

1.00 

.42 

.46 

Overa 11  £f f .  ( Supv ) 

.13 

.38 

.42 

1.00 

.83 

ELS  Ccmp.  (Supv) 

.15 

.40 

.46 

.83 

1.00 

HO  Total 

.12 

.  16 

.22 

.19 

.24 

JK  Total 

.06 

.  16 

.21 

.19 

.24 

Articles  15 

-.01 

-.21 

-.21 

-.23 

-.21 

Physical  Readiness 

.13 

.13 

.11 

.11 

.09 

Promotion  Rate 

.08 

.21 

.21 

.22 

.21 

ASVAB  Factors 

Quantitative 

.04 

.07 

.10 

.10 

.14 

Speed 

.02 

.08 

.07 

.09 

.08 

Technical 

.07 

.11 

.15 

.10 

.15 

Verba  I 

LVl  Sample 

.02 

.03 

.07 

.04 

.08 

ELS  Comoonents 

■Awards 

1.00 

.14 

.18 

.15 

.  16 

Overall  Eff.  (Peer) 

.14 

1.00 

.78 

.36 

.36 

ELS  Como.  (Peer) 

.18 

.78 

1.00 

.40 

.42 

Overall  Etf.  (Supv) 

.  15 

.36 

.40 

1.00 

.84 

ELS  Comp.  (Supv) 

.16 

.36 

.42 

.84 

1.00 

HO  Total 

.08 

.14 

.17 

.22 

.23 

JK  Total 

.08 

.  1  0 

.15 

.24 

.25 

Articles  15 

.07 

.21 

.22 

.24 

.22 

Physical  Readiness 

.  i  5 

.13 

.14 

.11 

.11 

Promotion  Rate 

.10 

.27 

.27 

.30 

.29 

'jSVAB  Factors 

Ouant 1  tat i ve 

.07 

.08 

.10 

.13 

.14 

Speed 

.03 

.08 

.04 

.06 

.06 

Technical 

.07 

.13 

.17 

.16 

.19 

Verba  1 

.06 

.06 

.09 

.07 

.08 

•Note.  Correlations  are  averaged  across  'lOS. 
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tieans,  Effect  Sizes,  and  Ceiling  Effects  for  ABLE  Scale  and  Factor-Based  Scores,  CVJ  and  LVI 
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LVI  differences.  In  contrast,  there  were  no  differences  between  the  samples 
cn  Oominance,  and  the  CVI  sample  outscored  the  LVI  sample  on  the  Physical 
Cond’tion  scale. 

The  pattern  of  CVI  to  LVI  differences  in  means  on  the  content  scales  is 
qiiite  different  from  those  observed  during  the  Trial  Battery  faking  experiment 
(i.e.,  the  comrarison  between  Honest  vs.  Fake  Good  experimental  instructions) 
ccnductod  during  the  Trial  Battery  field  tests  (Peterson,  1987).  In  the 
Faking  Good  condition,  the  participants  changed  in  the  positive  direction  on 
all  the  scales  at  aoout  the  same  magnitude,  wfiereas  CVI  versus  LVI  differences 
vary  Dy  scale.  Dominance,  for  example,  was  strongly  faked  good  (effect  size  = 
.70)  in  the  faking  study,  while  there  no  difference  between  CVI  and  LVI  on 
th-is  scale. 

A  'ceilicig  effect'  occurs  when  most  people  obtain  high  scores  on  a 
tes:.  For  ti'.e  CVI  sample  the  only  scale  with  a  ceiling  effect  was  Physical 
Condition,  ^or  the  LVI  data,  the  iacgeit  ceiling  effects  occur  for 
Trad i t  ior a  1  Values  and  Internal  Control,  and  from  the  factor-based  scores,  for 
Locus  cf  Control.  In  ^hori,  vaniance  is  attenuated  on  these  scales. 

The  cerre I  at  ions  between  Social  Desirability  and  tne  content  seal'  '  are 
also  hiahar  fer  the  !.  V !  r.amole  by  about  .10.  Consider,  for  example,  the  168- 
iten  f cctcr-tased  score  version  of  the  ABLE.  Correlations  between  Social 
Desirability  and  the  content  scales  for  the  CVI  sample  ranged  from  .08  to  .34 
with  a  mean  of  .20;  for  the  LV!  sample,  these  correlations  ranged  from  .16  to 
.42  with  a  Hiean  of  .29  (see  Table  7,25). 


Table  7.25 

Correlations  Between  /,3LE  Factor-Based  Scores  and  ABLE  Social 
Des  i  rdb  i '  i  ty_^a  1  e _ 


able-  ABLE- 

168  Items  114  Items 


CV  Sarrola  Range  o'-  r  .0c-.34  .08-.  31 

Mean  r  .20  .18 

LV  Samr/ie  Range  of  r  .16-. 42  .16-. 4^^ 

Moanr 


We  also  comoared  CVI  and  LVI  intercori-e  l.Jt  ions  for  a  variet'/  of  sets  of 
ABiE  scores:  (o'  cemoos :  tes  forme'.:  us  wig  TV  njlos,  ;t)  composites  forir.ed 
us'ng  LV  rules,  and  (c)  composites  formed  csing  the  two  factor-Dased  Si.or  ing 
keys.  Regardless  of  the  scoring  ,mt‘tlsod  useo,  i.Vl  c;M'''eldt  ions  are  abc 't  .06 
to  .10  higher  than  those  from  CV;  oaio  (see  Tables  7.26  -  7.29.) 


Correlations  Between  ABLE  Composites  Computed  Kith  CV  and  LV  Rules  on  a  Longitudinal  Validation  Sample 
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Aii.  evemeni  Oienialion  includes  Sell  Esteem,  Wbrk  Oientalion,  and  Energy  Level  Leadership  Potential  includes  only  Dominance  Dependability  includes 
Tiailitional  Values  Conscientiousness  and  Nondelinquency  Ad|uslnnent  includes  Emotional  Stability  Cooperativeness.  Internal  Control,  and  Physical  Condition  each 
subs  jme  only  one  scale,  the  scale  with  Ihe  same  name 


Correlations  Between  ABLE  Factor-Based  Scores  on  CV  Data 
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Correlations  Between  ABLE  Factor-Based  Scores  on  LV  Data 
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A  Limited  Conclusion 


In  general,  the  somewhat  lower  correlation  of  ABLE  with  Effort/Leadesh ip 
in  LVI  seems  due  to  the  joint  effects  of  two  influences.  First,  the 
determinants  of  ELS  scores  seem  to  favor  ability  slightly  more  and  motivation 
slightly  less  in  LVI  versus  CVI,  perhaps  because  their  true  score  variances 
are  different  across  the  two  cohorts.  Second,  the  greater  influence  of  the 
social  desirability  response  tendency  in  LVI  seems  to  produce  more  positive 
manifold  (i.e.,  higher  intc.'corre i at’ons  for  the  LVI  ABLE  basic  scores),  as 
contrasted  with  CVI.  This  could  also  lower  the  correlation  of  the  regression- 
weighted  ABLE  composite  with  ELS.  whereas  it  might  not  have  the  same  effect 
with  the  Core  Technical  and  General  Soldiering  factors. 

Yet  another  component  of  the  explanation  is  the  negative  correlation 
between  the  Social  Desirability  scale  and  AFQT.  As  shown  in  Table  7,30,  the 
high  Social  Desirability  responders  tend  to  have  lower  AFQT  scores.  AFQT  and 
Social  Desirability  correlated  -.22  in  the  CV  sample  and  -.20  in  the  LV 
sample.  This  would  tend  to  lower  the  correlation  between  ABLE  and  ELS  if  the 
correlations  between  ABLE  and  ASVAB  and  between  ASVAB  and  ELS  are  positive, 
which  they  are. 

The  above  conclusions  must  remain  tentative  because  the  effects  are 
small  and  correlation  does  not  imply  causation.  Obtaining  a  definitive  answer 
would  require  the  experimental  manipulation  of  both  performance  determinants 
and  response  sets,  which  is  beyond  the  scope  of  this  report. 


Table  7.30 


Mean  AFQT  Scores  for  Accurate  and  High  Social  Desirability  Responders  in  a 
Longitudinal  Validation  Sample _ 


MOS 

All 

Accurate^ 

Hiah^ 

N 

Mean 

N 

Mean 

N 

Mean 

lie 

235 

60.02 

174 

61.59 

61 

55. 54 

13B 

551 

47.15 

355 

50.02 

196 

41.96 

19K 

445 

60.96 

305 

62.80 

140 

56.94 

31C 

172 

64.24 

136 

66.47 

36 

55.81 

63B 

406 

50.72 

296 

53.03 

no 

44.50 

71L 

251 

55.21 

180 

56.70 

71 

51.44 

88M 

221 

46.64 

162 

48.61 

59 

41.24 

91A 

535 

62.05 

380 

63.78 

147 

57.50 

95B 

270 

62.23 

211 

62.93 

59 

59.73 

^Social  Desirability  score  less  than  18.2. 

'^Social  Desirability  score  greater  than  or  equal  to  18.2. 
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SUHHARY  AND  CONCLUSIONS 


The  preceding  analyses  of  basic  validation  results  for  the  LVI  sample 
produced  a  number  of  noteworthy  findings  in  relation  to  the  objectives  that 
guided  this  round  of  analyses. 

Generally  speaking,  the  ASVAB  was  the  best  predictor  of  performance. 
However,  the  composite  of  spatial  tests  provided  a  small  amount  of  incremental 
validity  for  the  "can  do"  criteria  0*3  points),  and  the  ABLE  provided  larger 
increments  (7-20  points)  for  two  cf  the  three  "will  do"  criteria  (Maintaining 
Personal  Discipline,  and  Physical  Fitness  and  Bearing).  Note  that  the 
estimates  of  incremental  validity  were  somewhat  higher  when  the  results  were 
not  corrected  for  range  restriction. 

With  regard  to  the  ASVAB  and  options  in  using  subtest  scores  to  form 
prediction  equations,  results  here  indicate  highly  similar  results  across  four 
methods,  with  a  very  slight  edge  going  to  multiple  regression  equations  using 
the  four  ASVAB  factor  scores  in  the  equation.  These  factors  are  unit-weighted 
composites  of  the  ASVAB  subtests. 

In  the  test  of  several  ABLE  scoring  options,  the  method  using  factor 
scores  computed  from  a  subset  of  a  1 1  the  ABLE  items  proved  to  have  generally 
higher  validities,  but  the  difference  was  not  large.  The  availability  of  a 
version  that  is  at  least  as  valid,  but  shortened,  of  this  biodata/temperament 
instrument  is  gratifying. 

In  the  comparison  of  sample  editing  strategies,  the  use  of  setwise 
deletion  of  examinees  (versus  listwise)  substantially  increased  the  samples 
available  for  validation  analyses.  The  validation  results  showed  similar 
patterns  across  the  two  methods,  but  the  coefficients  computed  on  the  setwise 
samples  were  about  one  tc  three  points  higher. 

Perhaps  the  most  interesting  finding  in  this  evaluation  of  predictors 
derived  from  the  comparisons  between  the  Longitudinal  Validation  results  and 
these  from  the  Concurrent  Validation.  Such  comparisons  are  rarely 
accomplished,  because  it  is  extremely  difficult  to  conduct  a  concurrent 
validation  and  longitudinal  validation  study  in  the  same  organization  on  the 
same  set  of  varied  jobs  using  essentially  the  same  predictors  and  criteria. 

The  rarity  increases  because  the  sets  of  both  predictors  and  criteria  used  in 
the  present  research  are  both  very  comprehensive  and  carefully  developed  and 
the  achieved  sample  sizes  are  impressive.  Aside  from  the  concurrent  versus 
longitudinal  design  difference,  only  cohort  differences  (both  examinees  and 
raters/scorers)  can  explain  any  disparities  in  the  validation  results. 

Generally  speaking,  the  pattern  and  level  of  the  validity  coefficients 
are  highly  sirn'lir  across  the  two  samples.  The  correlation  between  the  CV  and 
LV  coefficients  in  i.  jle  7.20  is  .y62  and  the  root  mean  squared  difference 
between  the  two  ■'ets  of  coefficients  is  .046. 

Note,  however,  that  the  correlation  is  not  1.00,  nor  is  the  RMS 
difference  zero.  As  we  noted  above,  the  longitudinal  validities  are  higher 
for  cognitive  predictors  against  "will  do"  criteria  and  lower  for  ABLE 
composites  against  "will  do"  ci'itena.  Some  of  the  possible  explanations  for 
those  differences  include  changes  m  the  nature  of  predictor  scores  when 
administered  in  a  longitudinal  versus  concurrent  design,  changes  in  criterion 
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or  predictor  scores  due  to  cohort  differences,  and  changes  in  the  true 
relationship  between  abilities  and  performance  as  persons  gain  more  experience 
and  training  in  an  organization  and  job.  These  and  other  possible  explana¬ 
tions  will  be  explored  in  future  analyses. 


193 


Chapter  8 

RESULTS  OF  THE  SECONO-TOUR  VALIDATION  (CVII) 

John  P.  Campbell  and  Jeffrey  W.  Johnson 

This  chapter  is  a  report  on  the  validity  of  ASVAB  and  ABLE  for 
predicting  performance  as  a  noncommissioned  officer  (NCO)  in  the  second  tour 
of  duty.  The  results  are  based  on  the  CVII  sample  which,  as  per  the  original 
design  of  Project  A,  was  assessed  on  the  criterion  measures  of  second-tour 
performance  at  the  same  time  that  the  LVI  performance  data  were  collected  from 
the  first-tour  longitudinal  sample. 

The  predictor  set  is  limited  to  ASVAB  and  ABLE  because  only  a  small 
proportion  (approximately  12  percent)  of  the  CVII  sample  has  been  assessed 
with  the  Experimental  Predictor  Battery.  ASVAB  scores,  taken  5-6  years 
earlier,  were  available  from  the  Enlisted  Master  File.  The  ABLE  was 
administered  currently  during  the  CVII  data  collection  to  approximately  45 
percent  of  the  total  sample  (i.e.,  those  individuals  who  had  no  peers  in  the 
sample  to  rate  and  thus  had  time  to  take  the  ABLE).  Everyone  in  the  sample 
was  assessed  on  the  full  set  of  second-tour  performance  measures.  By  design, 
the  MOS  in  the  CVII  sample  were  limited  to  the  MOS  in  Batch  A. 

Because  of  the  generally  small  samples  for  individual  MOS,  results  for 
most  analyses  are  reported  for  the  combined  sample.  Selected  results  are 
reported  for  clusters  of  MOS  (combat,  technical,  and  support). 


OBJECTIVES 

The  objectives  of  the  CVII  validation  were  as  follows: 

(1)  Compute  the  basic  validities  for  ASVAB  and  ABLE 
against  the  performance  factor  scores  and  selected 
individual  performance  measures. 

(2)  Compute  the  incremental  validities  for  ABLE  over 
ASVAB. 

(3)  Compare  the  validities  of  three  alternative  sets  of 
ABLE  scores. 

(4)  Compare  the  results  of  using  the  four  ASVAB  factors 
vs.  the  nine  ASVAB  subtests  in  a  weighted  predicted 
composite. 

(5)  Evaluate  the  effects  on  validities  of  using  unit 
weights. 

(6)  Using  scores  from  ASVAB  and  ABLE,  determine  the  "optimal" 
prediction  equation  for  each  performance  factor. 

(7)  Determine  the  genera  1 izabi 1 ity  of  prediction  equations 
across  performance  factors  (using  the  combined 
sample) . 
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(8)  For  each  performance  factor,  determine  the  general iz- 
ability  of  the  optimal  prediction  equation  in  a 
specific  MOS  cluster  to  other  MOS  clusters. 


THE  SAMPLE 

The  evil  data  collection  and  data  preparation  are  described  in  the  first 
annual  report  for  Building  the  Career  Force  (Campbell  &  Zook,  1990;  see 
Chapters  5  and  6).  After  final  editing,  the  total  N  for  CVII  was  1,053.  The 
total  sample  was  distributed  across  the  Batch  A  MOS  as  shown  in  Table  8.1. 

Table  8.1 


CVII  Sample  Sizes  by  HOS 


MOS 

N 

118 

Infantryman 

127 

13B 

Cannon  Crewman 

162 

19E 

M60  Armor  Crewman 

33 

19K 

Ml  Armor  Cre'wman 

10 

31C 

Single  Channel  Radio  Operator 

103 

63B 

Light-Wheel  Vehicle  Mechanic 

116 

71L 

Administrative  Specialist 

112 

88M 

Motor  Transport  Operator 

144 

91A 

Medical  Specialist 

146 

958 

Mi  1  itary  Police 

141 

Total 

1,053 

Because  of  some  missing  data,  the  sample  sizes  varied  depending  on 
the  specific  analysis  being  reported.  For  example,  for  the  reasons  cited 
in  the  introduction,  ABLE  scores  were  available  only  for  477  individuals. 
Consequently,  all  the  analyses  that  require  a  common  covariance  matrix  for 
ABLE  and  ASVAB  are  based  on  this  reduced  sample. 


PERFORMANCE  CRITERION  MEASURES 

The  CVII  performance  measures  have  been  described  previously  (e.g., 
Campbell  &  Zook,  1990;  Campbell,  1991).  For  informational  purposes,  they  are 
listed  below. 

•  Revised  Hands-On  (HO)  job  sample  measures  (a  sample  of  Army-wide  and 
MOS-  specific  technical  tasks  that  had  been  developed  for  first-tour 
testing  and  was  revised  on  the  basis  of  the  second-tour  job  analyses). 

•  Revised  Job  Knowledge  (JK)  tests. 

•  The  leader  Situational  Judgment  Test  (SJT). 
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•  Revised  Army-wide  rating  scales  (BARS). 

•  Newly  developed  Army-wide  scales  for  the  critical  components  of 
leadership  and  supervision  that  were  identified  in  the  second-tour 
job  analyses. 

•  Revised  predicted  combat  performance  rating  scales  (males  only). 

•  Administrative  indices  of  performance. 

•  Three  simulation  (role-play)  exercises  of  approximately  20  minutes 
each,  designed  to  assess  performance  in  three  specific  face-to- 
face  leadership  situations  (performance  counseling,  personal 
counseling,  individual  remedial  training). 

These  measures  generated  a  set  of  20  basic  scores  that  were  the  basis 
for  the  evil  performance  modeling  analysis. 


THE  evil  PERFORMANCE  MODEL 

The  development  of  the  CVII  performance  model  has  been  described 
previously  and  is  summarized  in  the  Building  the  Career  Force  first  annual 
report  (Campbell  &  Zook,  1990).  The  solution  that  yielded  the  best  fit 
consisted  of  six  substantive  factors  and  two  methods  factors.  The  two  methods 
factors  were  defined  to  be  orthogonal  to  the  substantive  factors,  but  the 
correlations  among  the  substantive  factors  were  not  so  constrained.  The  six 
substantive  factors  and  two  methods  factors,  and  the  variables  that  are  scored 
on  each,  are  shown  in  Figure  8.1. 

A  number  of  suggested  modifications  were  evaluated  to  see  whether  they 
produced  a  better  fit.  For  example,  Lloyd  Humphreys  (personal  communication, 
1990)  suggested  that  the  Personal  Discipline  (MPD)  factor  was  really  not  very 
distinct  and  perhaps  should  not  constitute  a  separate  score.  Our  own 
exploratory  factor  analyses  concurred  with  his  in  that  the  factor  solutions 
provide  little  evidence  for  such  a  factor.  However,  the  a  priori  model  that 
included  the  factor  provided  a  slightly  better  fit  in  the  confirmatory 
analysis  than  one  that  did  not  include  it  and  we  elected  to  keep  it  as  a 
separate  performance  factor,  at  least  for  some  of  the  initial  analyses. 

A  principal  difference  between  the  exploratory  and  confirmatory  analyses 
is  that  the  two  orthogonal  methods  factors  were  part  of  the  a  priori  model 
specif ica-tions  for  the  confirmatory  analysis.  We  are  also  mindful  of  the 
fact  that  even  if  tw.  variables  intercorrelate  .95  they  can  still  have  very 
different  correlations  with  a  third  variable  (McCormack,  1956).  Consequently, 
wnile  the  MPD  factor  probably  doesn't  mean  the  same  thing  in  second  tour  as  it 
did  in  first  tour  and  its  variance  may  be  considerably  less  (i.e.,  many  of  the 
people  who  get  in  trouble  are  gone),  and  it  is  difficult  to  distinguish  from 
the  effort/Leadership  (ELS)  factor,  it  is  analyzed  as  a  separate  criterion 
factor  in  the  basic  validation.  The  composite  of  ELS  and  MPD  is  also  used  as 
a  criterion  factor.  Thus,  we  "did  it  both  ways." 

Another  interesting  --esult  of  the  modeling  analysis  was  that  the 
Situational  Judgment  Test  fit  best  in  the  ELS  factor,  as  it  theoretically 
should,  even  though  it  is  a  paper-and-penc i 1  measure.  Also,  when  the 
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intercorrelations  among  the  criterion  factor  scores  are  examined,  the  role- 
play  factor  clusters  with  the  proficiency  (CTP  and  GSP)  factors  to  form  a 
higher  order  "can  do"  factor  even  though  the  scoring  system  depends  on 
observed  ratings. 


•  Core  Technical  Proficiency  (CTP) 

-  Hands-On  Test  -  MOS-Specific  Tasks 

-  Job  Knowledge  Test  -  MOS-Specific  Tasks 

•  General  Soldiering  Proficiency  (GSP) 

Hands-On  Test  -  Common  Tasks 

-  Job  Knowledge  Test  -  Common  Tasks 

•  Effort  and  Leadership  (ELS) 

-  Admin.  Index  -  Number  of  Awards  and  Certificates 

-  Admin.  Index  -  Number  of  Training  Courses  Taken 

Admin.  Index  -  Promotion  Grade  Deviation  Score 

-  Army-Wide  BARS  Leadership  Ratings  Factor 

-  Army-Wide  BARS  Technical  Ratings  Factor 

-  Average  of  MOS  BARS  Ratings  Scales 

-  Situational  Judgment  Test  Total  Score 

•  Maintaining  Personal  Discipline  (MPO) 

-  Admin.  Index  -  Number  of  Articles  15,  Flag 
Actions  (reversed) 

-  Army-Wide  BARS  Personal  Discipline  Rating  Scale 

•  Physical  Fitness/Military  Bearing  (PFB) 

-  Admin,  Index  -  Physical  Readiness  Score 

-  Army-Wide  BARS  Fitness/Bearing  Ratings  Factor 

•  Training  and  Counseling  Subordinates  (TCS) 

-  Simulation  Exercise  -  Personal  Counseling  Content 

-  Simulation  Exercise  -  Personal  Counseling  Process 

-  Simulation  Exercise  -  Disciplining  Content 

-  Simulation  Exercise  -  Disciplining  Process 

-  Simulation  Exercise  -  Training 


•  Written  Methods  Factor  (WM) 

-  Job-Specific  Knowledge 

-  Common  Soldiering  Knowledge 

-  Situational  Judgment  Test 

•  Ratings  Method  Factor  (RM) 

-  Four  Army-Wide  BARS  Ratings  Factors 

_ -  MOS  BARS  Average  _ 

Figure  8.1.  The  latent  variables  in  the  CVIl  Performance  Model  and  the  basic 
criterion  scores  that  define  them. 
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The  complete  basic  validation  analyses 
the  performance  factors,  as  shown  below. 


(1) 

TCS 


(2) 

CTP 


(3) 

GSP 


(4) 

ELS 


utilize  a  total  of  10  scores  for 

(5)  (6) 

MPD  PFB 


(7)  (CTP 

+  GSP) 

(8)  (ELS)  +  MPD) 

(9)  (TCS  +  C 


P  +  GSP) 


10)  (ELS  +  MPD  +  PFB) 


That  is,  all  10  scores  are  used  as  criterion  measures.  All  higher  or-der 
composite  scores  were  obtained  by  standardizing  the  component  scores  and  then 
taking  the  simple  sum. 


VALIDATION  PROCEDURE 

The  evil  validation  analysis  procedure  consisted  of  the  following  steps 

A)  The  ASVAB  and  ABLE  were  correlated  with  the  six  performance 
factor  scores,  their  five  residual  scores  (there  was  no 
residual  for  TCS),  the  higher  order  factor  composites,  the  two 
methods  factor  scores,  and  the  total  score  from  the  Hands-On 
measure,  the  Job  Knowledge  Test,  and  the  Situational  Judgment 
Test. 

1)  ASVA8  was  represented  by 

a)  The  AFQT. 

b)  A  regression-weighted  composite  of  the  four  factors. 

c)  A  regression-weighted  composite  of  the  nine  subtests. 

2)  ABLE  was  represented  by  three  alternative  sets  of  scores. 

a)  The  seven  rational/theoretical  scales  developed  for  the 
Experimental  Battery. 

b)  The  seven  empirical  scales  developed  on  the  basis  of  a 
factoi’  analysis  that  retained  all  the  items  (168  were 
used).  This  analysis  is  described  in  Chapter  2  of  this 
report. 

c)  The  seven  empirical  scales  developed  by  using  the 
results  of  the  factor  analysis  to  select  the  best  items 
to  reflect  each  factor;  114  items  were  retained.  This 
analysis  is  also  described  in  Chapter  2  of  this  report. 

Both  corrected  (for  multivariate  restriction  of  range)  and 

uncorrected  estimates  were  computed,  and  both  regression 
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weights  and  unit  weights  (applied  to  standard  i.red  scores) 
were  used.  When  multiple  regression  weights  were  used,  the 
Rczeboom  correction  (Rozebooin,  1978)  was  used  to  account 
for  the  fitting  of  error. 

B)  As  in  CVI,  incremental  validities  for  the  ABLE  composites  over 
the  ASVAB  composites  were  also  computed  against  each  crite/ion 
score. 

C)  A  hierarchical  regression  analysis,  stopping  at  six 
predictors,  was  run  against  each  performance  factor,  factor 
composite,  and  individual  criterion  score  (i.e.,  HO,  JK,  and 
SJT).  Only  the  four  ASVAB  factors  and  the  seven  empirica’ 

ABLE  factors  based  on  114  items  were  used  as  predictors.  For 
reasons  that  will  be  explained,  this  combination  seemed  to  be 
the  best  set  of  scores;  consequently,  there  are  a  total  of  11 
predictors.  The  step-wise  procedure  was  stopped  at  six 
because  virtually  no  additional  information  was  obtained  after 
that  point.  Solutions  were  computed  for  both  the  corrected 
and  uncorrected  covariances.  The  final  multiple  correlation 
was  adjusted  using  the  Rozeboom  correction. 

D)  A  hierarchical  regression  analysis  was  also  carr't.d  out  on 
selected  criterion  variables  for  the  combined  sa..  )1es  from 
three  MOS  clusters.  The  clusters  were  based  on  the  results  of 
an  MOS  clustering  within  the  Synthetic  Validation  Project 
(Wise  et  al.,  1991)  and  on  tlie  results  of  the  validity 
generalization  analysis  for  the  Batch  A  MOS  in  the  CVI  sample 
(Wise,  McHenry,  &  Campbell,  1990).  The  three  clusters  are: 

Cluster  One:  Combat  soecialties  (MOS  ilB,  13B,  19L, 

95B/N  =  218) 

Cluster  Two:  Technical  specialties  (MOS  31C,  63B/N  =  96) 

Cluster  Three:  Support  specialties  (MOS  HL,  91A/N  =  111) 

MOS  88M  was  not  included  because  its  cluster  membershio  was 
less  clear.  The  objective  here  was  to  maximize  a  prion  the 
opportunity  for  detecting  differential  validity  within  what  is 
essentially  a  field  test  sample.  The  cluster  memberships  are 
not  cast  in  stone. 

E)  The  final  step  consisted  of  using  the  optimal  six  variable 
equations  from  the  hierarchical  regression  analyses  described 
ab've  to  develop  a  picture  of  the  degree  uf  differential 
prediction  across  performance  factors  and  across  the  three 
MOS  clusters. 

])  To  examine  differential  prediction  across  perform¬ 
ance  factors  and  factor  composites  on  the  combined 
sample,  a  square  matrix  of  multiple  correlations  was 
generated  by  applying  the  regression  equation 
developed  for  a  specific  criterion  variable  to  every 
other  criterion  variable.  The  diagonal  was  adjusted 
using  the  Rozeboom  correction. 
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2)  To  examine  differential  prediction  across  the  three  MOS 
clusters,  a  3  X  3  matrix  of  correlations  was  generated  for 
each  of  the  six  performance  factors  and  four  factor 
composites.  For  each  matrix,  the  equation  developed  for  a 
specific  MOS  cluster  was  applied  to  the  data  from  each  of 
the  other  two.  The  diagonal  vras  adjusted  using  the 
Rozeboom  correction. 


RESULTS 

For  r'efe’“ence  purposes,  the  full  list  of  predictor  and  criterion 
variables  and  their'  des-ignaters  is  grvtn  in  Table  8.2.  The  intercorre lations 
of  trie  19  criterion  scores,  based  on  the  combined  sample,  are  given  in  Table 

8.3. 


Table  8.2 

Variable  Hames  for  ASVAB  Factors,  ABLE-114  Factors,  and  19  Criterion  Vai  rabies 


Var i dO  ie 


Label 


Fred i cters 
AlATLCH 
AlAQUANT 
AlAVERBL. 
AIASPEEO 
e8FSCh^2 
P3FSCaP2 
B8FS0MN2 
B8FSDPN2 
B8FSPCD2 
B8FSSTB2 
85FSWRK2 


Composite:  Technical 
Composite:  Quantitative 
Composite:  Verbal 
Composite:  Speed 

Score;  Locus  of  Ccnfol  (114  items) 
Score:  Cooperation  (114  items) 

Score:  Dominance  (114  items) 

Score:  Dependability  (114  items) 
Score:  Physical  Condition  (114  items) 
Score:  Stress  Tolerance  (IK  items) 
ABLE  Factor  Score:  Work  Orientatiofi  (114  i^ems) 


ASvA?  Factor 
ASVAB  Factor 
ASVAB  Factor 
ASVAB  Factor 
ABLE  Factor 
ABLE  Factor 
able  Factor 
ABLE  Factor 
Factor 
Factor 


ABLE 

ABLE 


Criteria 

MSRAWCiP  Core  Techical  Proficiency,  CVII  Raw  Construct  (STD) 

M8RAWGSP  General  Soldier  Proficiency,  CVII  Raw  Construct  (STD) 

MSRAWElS  Effort/Leadership,  CVII  Raw  Construct  (STD) 

ELSNOSJT  Effort/l  eadership ,  Situational  Judgment  Test  (SJT)  removed  (STDl 
MBRAWMPD  Personal  Discipline,  CVII  Raw  Construct  (STD) 

MSRmWPFb  phys’Cd'  Fitness,  CVII  Raw  Construct  (STD) 

M6RAw.  ''S  Train/Ccunsel ,  CVII  Raw  Construct  (STD) 

MSREsClP  Core  Te'^hmca'  Proficiency,  CVII  Residual  Score  (STD) 

M8RESGSP  Genera!  Soldier  Proficiency,  CVII  Residual  Score  (STD) 

M8RESELS  Ef  fort/ Leadesh  ,  p ,  C'.'II  Residi.al  Score  (STD) 


(Continued) 
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M8RESMP0 

M8RESPFB 

M8CTPGSP 

M8ELSMPD 

CRITCGTl 

CRITEMP2 

M8XHT0TT 

M8XKT0TT 

M8XST0T 


Personal  Discipline.  CV!I  Residual  Score  (STD) 
Physical  Fitness,  CVII  Residual  Score  (STD) 
Criterion  Composite:  CTP  (Raw)  +  GSP  (Raw)  (STD) 
Criterion  Composite:  ELS  (Raw)  +  MPD  (Raw)  (STD) 
Criterion  Factor  1:  CTP+6SP+TCS 
Criterion  Factor  2;  ELS+MPD+PFB 
Hands-On  Average  Percent  GO  for  All  Tasks  (STD) 
Job  Knowledge  Total  Percent  Correct  Score  (STD) 
Situational  Judgment  Test  Most  -  Least  (STD) 


Table  8.3 


Intercorrelations  of  19  Criterion  Scores  Based  on  the  Combined  CVII  Sample 


KSRAUCTP 

T^SRAUCSP 

M8RJ117EL3 

KBRjkUHPO 

M8RAUPFB 

M8RAUTCS 

MSRESCr? 

H8RESCSF 

M8RESELS 

“8RESMPD 

M6RAUCTP 

1  .00 

.  59 

.  39 

.16 

-.01 

.  24 

.91 

.39 

.  2  7 

.  16 

MeHAWOSP 

.56 

1 .00 

.63 

.11 

-.03 

.26 

.36 

.90 

.30 

.12 

M8RAur;.s 

-39 

.93 

1.00 

.95 

.29 

.26 

.26 

.  29 

.85 

.26 

MeR/lWO 

.  16 

.  1 1 

.  6' 

1 .00 

.33 

.07 

.16 

.25 

.86 

nafuvp.'B 

.  .0; 

-  .0! 

.  29 

.33 

1.00 

.02 

.02 

-.03 

,  .3 

. ;  7 

M8BJIUTCS 

.26 

.26 

.07 

.02 

1.00 

.25 

.28 

.31 

.  iO 

M8P.E5CTP 

.9i 

.  36 

.  26 

.16 

.02 

.23 

1  .00 

.38 

.25 

.13 

MSRtSCSP 

.39 

.90 

.29 

.10 

-.03 

.28 

.38 

1.00 

.28 

.08 

M8R15ELS 

.27 

.30 

.83 

.23 

.18 

.31 

.23 

.28 

1.00 

.29 

MRRESKI'O 

.16 

.  !2 

.26 

.89 

.  1 7 

.13 

• 

.08 

•  4  r 

l.OC 

H^RCSPFil 

-.02 

..06 

.19 

.13 

.91 

.03 

-.02 

1 

o 

.  19 

.19 

ELSNOSJT 

.31 

.29 

.88 

.31 

,90 

.22 

.32 

.30 

.  76 

.29 

MSCTPCSP 

.87 

.88 

.47 

.13 

-.03 

.29 

.72 

.  79 

.33 

.16 

MBELSKPO 

.33 

.32 

.86 

.84 

.38 

.20 

.23 

.29 

.66 

.63 

critcct; 

.80 

.81 

.98 

.13 

-.01 

.63 

.68 

.70 

.39 

.17 

CRITtKPJ 

.29 

.22 

.78 

.78 

.72 

.  16 

.19 

.17 

.  3? 

.55 

p!«XKTorr 

.65 

.66 

.31 

.11 

.01 

.22 

.69 

.  72 

.33 

.11 

MsxRTorr 

.73 

.80 

.93 

.14 

-.07 

.27 

.95 

.53 

.  26 

.19 

M8XSTCIT 

.  39 

.92 

.71 

.16 

.00 

.20 

.05 

.  14 

.58 

.16 

H8RESPFB 

ELSNOSJT 

HBCTPCSP 

M8ELSMP0 

CRITCCTl 

CRITniP2 

.'^8XHT0T: 

MSXKTOTT 

^^^^XSTOT 

M8P01WCTP 

-.02 

.31 

.87 

.33 

.80 

.29 

.65 

.73 

.34 

M8P.AWCSP 

-.06 

.29 

.88 

.32 

.81 

.22 

.66 

.80 

> 

MRiUUELO 

.  1  9 

.88 

.97 

.86 

.98 

.78 

.31 

.98 

. .  1 

.“.8rjiVi’?o 

.  15 

.31 

^  ;  s 

.89 

.5 

.  78 

.  1 1 

.  6 

.  ;  i 

nePowpFB 

.91 

.90 

-  .03 

.38 

-.  ■'1 

.72 

.  31 

-.07 

.00 

M8R/UTCS 

.03 

.22 

.29 

.20 

.65 

.  16 

.22 

.  27 

.  20 

MSRESCrP 

-.02 

.32 

.  72 

.23 

.68 

.19 

.69 

.95 

.05 

M8RESCSP 

-.06 

.30 

.  79 

.29 

.70 

.17 

.22 

.53 

,19 

.“isresel: 

.  19 

.  76 

.33 

.66 

.39 

.it 

.33 

.29 

.58 

M8RESMPD 

.19 

.29 

.  1 6 

.63 

.17 

.55 

.  1 1 

.19 

.16 

H8RESFFB 

1.00 

.  19 

-.05 

.17 

-.02 

.53 

.00 

-.10 

-  .0! 

ELSllOS-'T 

.  19 

1.00 

.39 

.83 

.37 

.79 

.26 

.32 

.29 

M8CTPCSP 

-  .0.3 

.39 

1.00 

.37 

.92 

.27 

.  79 

.88 

.-3 

MSELSffD 

. :  7 

.83 

.37 

1 .00 

.38 

.91 

.23 

.32 

.51 

CRITCCTl 

-.02 

.37 

.92 

.38 

1.00 

.28 

.68 

.81 

.93 

cr;temp2 

.53 

.  79 

.27 

.91 

.28 

1  .00 

.  19 

.25 

.38 

M8XHT0TT 

.00 

.  26 

.  79 

.25 

.68 

.19 

1.00 

.  ..9 

''  1 

MBXETorr 

-  .  10 

.32 

.88 

.37 

.81 

.23 

.99 

1.00 

.49 

M8XST0T 

-.01 

,29 

.93 

.31 

.93 

.38 

.23 

,99 

1.00 

Note. 

Corrected 

for  range  restriction 

See 

Table  8 

2  for 

var  i  db  le 

names . 
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Basic  Validities 


The  basic  multiple  correlations  for  ASVAB  (4  factors  vs.  9  subtests) 
and  ABLE  (7  theoretically  based  composites  vs.  7  "purified"  empirical  factors) 
are  given  in  Tables  8.4  and  8.5.  The  combined  sample  for  Table  8.5  is  smaller 
because  less  than  half  the  sample  took  the  ABLE  during  the  CVII  data 
col  lection. 

Several  things  are  worth  noting.  ASVAB,  taken  at  time  of  entry,  is 
still  a  highly  valid  p’^edictor  of  Core  Technical  and  General  Soldiering 
Proficiency  and  has  respectable  validity  for  Effort/Leadership.  For  ASVAB, 
the  four  factors  and  the  nine  subtests  provide  virtually  the  same  level  of 
predictive  accurac''.  However,  for  ABLE  the  reduced  factor  scores  (114  items) 
are  consistently  the  best  predictor  set.  ABLE  predicts  Effort/Leadership  and 
Physical  Fitness  very  well  and  has  reasonable  correlations  with  General 
Soldiering  and  Training/Counseling.  Personal  Discipline  is  not  well  predicted 
by  anything,  which  is  consistent  with  the  previous  discussion. 


Table  8.4 


Multiple  Correlations  for  4  ASVAB  Factors, 
19  Criterion  Variables 

9  ASVAB 

Subtests , 

and  AFQT  Against 

Variab le 

ASVAB 

Factors 

ASVAB 

Subtests 

AFQT 

M8RAWCTP 

Core  Technical 

851 

.44 

.43 

.40 

M8RAWGSP 

General  Soldiering 

737 

.54 

.56 

.49 

M8RAWELS 

Ef f ort/Leadersh ip 

851 

.39 

.38 

.37 

M8RAWMP0 

Personal  Discipline 

851 

.00 

.00 

.01 

M8RAWPF8 

Physical  Fitness 

851 

.17 

.16 

.17 

M8RAWTCS 

Training/Counsel ing 

851 

.19 

.17 

.21 

M8RESCTP 

Core  Techn ica 1 

8J1 

.26 

.25 

.23 

M8RESG5P 

General  Soldiering 

737 

.38 

.40 

.33 

M8RESELS 

Eff ort/Leadersh ip 

851 

.25 

24 

.23 

M8REFMPD 

Personal  Discipline 

851 

.00 

.00 

.00 

M8RESPFB 

Physical  Fitness 

851 

.21 

.21 

.21 

ELSNOSJT 

ELS  -  No  SJT 

SSI 

.22 

.21 

.20 

M8CTPGSP 

Crit  Comp  CTP/GSP 

737 

.55 

.56 

.50 

M8ELSMP0 

Crit  Como  ELS/MPD 

851 

.27 

.26 

.25 

CRITCGTl 

Crit  1  CTP*GSP+TCS 

737 

.53 

.53 

.49 

CRITEHP2 

Crit  2  ELS+MPO+PFB 

851 

.13 

.10 

.  .11 

M8XHT0TT 

Hands-On  Average 

851 

.31 

.32 

.29 

M8XKT0TT 

Job  Knowledge  Total 

851 

.61 

.62 

.56 

M8XST0T 

Situation  Judgment 

851 

.45 

.44 

.44 

flote.  Adjusted  (Rozeboom  formula)  and  corrected  for  range  restriction. 
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Unit  Weights 


The  validity  coefficients  obtained  by  using  unit  weights  are  also  shown  in 
Table  8.5.  In  general,  after  adjustments,  regression  weights  and  unit  weights 
for  ASVAB  yield  about  the  same  level  of  validity.  However,  regression  weights 
are  somewhat  bette’^  than  un’t  weights  for  the  seven  empirical  ABLE  factors. 
There  is  not  as  much  positive  manifold  among  the  ABLE  factors  as  there  is 
among  the  ASVAB  subtests. 

Table  8.5 


Multiple  Correlations  for  4  ASVAB  Factors,  9  ASVAB  Subtests,  7  ABLE 
Composites,  and  7  ABLE-114  Scores  Against  19  Criterion  Variables  (All  HOS), 
With  Unit  Weights _ 


Variab  1e 

ASV/IB 

factors 

ASVAB 

SuDtests 

ABLE 

Conposites 

ABLE-114 

M8RAWCTP 

Core  Technical 

43 

(42) 

43 

(43) 

15 

(14) 

20 

(15) 

M8RAWGSP 

General  Soldiering 

56 

(54) 

57 

(55) 

22 

(16) 

26 

(18) 

M8RAWELS 

Effort/Leadership 

38 

(38) 

39 

(38) 

37 

(32) 

41 

(32) 

M8RAWMP0 

Personal  Oiscipl ine 

00 

(11) 

00 

(11) 

20 

(21) 

18 

(22) 

M8RAWPEB 

Physical  Fitness 

13 

(16) 

06 

(16) 

32 

(23) 

34 

(21) 

M8RAWTCS 

Training/Counsel ing 

06 

(13) 

00 

(12) 

27 

(19) 

23 

(18) 

M8RESCTP 

Core  Technical 

29 

1291 

28 

(30) 

00 

(12) 

07 

(13) 

M8RESGSP 

General  Soldiering 

42 

(42) 

43 

(42) 

14 

(15) 

18 

(16) 

M8RESELS 

Effort/Leadership 

25 

(26) 

27 

(25) 

38 

(31) 

41 

(30) 

M8RESMP0 

Personal  Discipline 

00 

(09) 

00 

(09) 

16 

(20) 

15 

(19) 

M8RESPFB 

Physical  Fitness 

16 

(20) 

09 

(20) 

34 

(21) 

35 

(18) 

ELSNOSJT 

ELS  -  No  SJT 

24 

(22) 

23 

(22) 

34 

(31) 

38 

(30) 

M8CTPGSP 

Crit  Comp  CTP/GSP 

57 

(55) 

58 

(56) 

22 

(17) 

27 

(19) 

M8ELSMP0 

Crit  Comp  ELS/MPD 

29 

(?0) 

29 

(29) 

34 

(32) 

37 

(32) 

CRITCGT! 

Crit  1  CTP+GSP+-TCS 

50 

(50) 

50 

(50) 

29 

(22) 

32 

(23) 

CRITEMP2 

Crit  2  ELS+MPD*PFB 

14 

(16) 

12 

(15) 

34 

(35) 

35 

(34) 

M8XHT0TT 

Hands-On  Average 

39 

(40) 

38 

(40) 

12 

(12) 

18 

(13) 

M8XKTOTT 

.Job  Knowledge  Total 

59 

(56) 

59 

(57) 

25 

(14) 

28 

(16) 

M8XST0T 

Situation  Judgment 

42 

i43) 

42 

(43) 

27 

(20) 

31 

(21) 

Note.  N  =  412.  Adjusted  (Rozeboom  formula).  Validities  of  unit-weighted 
composites  are  in  parentheses.  Decimals  omitted. 


Incremental  Validities 

Table  8.6  contains  the  same  type  of  CVll  incremental  analyses  that  was 
done  in  CVI  (Campbell  &  Zook,  1991).  ABLE  does  not  add  to  the  prediction  of 
Core  Technical  and  General  Soldiering  Proficiency,  but  it  adds  about  the  same 
amount  to  the  prediction  of  Ef fort/Leadersn i p  as*  it  did  in  CVI.  However,  the 
overall  level  of  prediction  for  ELS  is  higher  in  CVII  than  it  was  in  CVI 
(i.e.,  .50  vs.  .43), 
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Table  8.6 


Multiple  Correlations  for  4  ASVAB  Factors  +  7  ABLE  Composites  and  +  7  ABLE-114 
Scores,  and  for  9  ASVAB  Subtests  *  7  ABLE  Composites  and  7  ABLE-114  Scores 
Against  19  Criterion  Variables,  All  HOS _ _ 


Variable 

ASVAB 
factors  « 
ABLE  Conp 
(K-11) 

ASVAB 
Factors  * 
ABLE-114 
(K-11) 

ASVAB 
Subtests  « 
ABLE  Coflip 
(K-16) 

ASVAB 
Subtests  ♦ 
ABLE-114 
(K.15) 

M8RAWCTP 

Core  Technical 

.42 

.43 

.42 

.43 

M8RAWGSP 

General  Soldiering 

.56 

.57 

.58 

.58 

M8RAWELS 

Effort/Leadership 

.49 

.49 

.49 

.50 

M8RAWMP0 

Personal  Discipline 

.16 

.13 

.09 

.03 

M8RAWPFB 

Physical  Fitness 

.34 

.35 

.32 

.33 

M8RAWTCS 

Training/Counseling 

.26 

.20 

.24 

.17 

M8RESCTP 

Core  Technical 

.24 

.26 

.24 

.25 

M8RES6SP 

General  Soldiering 

.42 

.42 

.44 

.44 

M8RESELS 

Effort/Leadership 

.43 

.43 

.43 

.43 

M8RESMP0 

Personal  Discipline 

.09 

.07 

.00 

.00 

M8RESPFB 

Physical  Fitness 

.36 

.37 

.34 

.34 

ELSNOSJT 

ELS  -  No  SJT 

.39 

.41 

.38 

.41 

M8CTPGSP 

Crit  Comp  CTP/GSP 

.57 

.57 

.58 

.58 

.^ISELSMPD 

Crit  Comp  ELS/MPD 

.40 

.40 

.40 

.40 

CRITCGTl 

Crit  1:  CTP^GSP+TCS 

.54 

.54 

.54 

.54 

CRlTtMP2 

Crit  2:  ELS^MPO+PFB 

.35 

.35 

.34 

.34 

M8XHT0TT 

Hands-On  Average 

.37 

.37 

.37 

.37 

M8XKT0TT 

Job  Knowledge  Tota  1 

.60 

.60 

.60 

.60 

M8XST0T 

Situation  Judgment 

.45 

.44 

.45 

.44 

Note.  N  =  412.  Adjusted  (Rozeboom  formula)  and  corrected  for  range 
restrict  ion . 


Hierarchical  Analyses 

The  hierarchical  analyses  on  the  combined  sample  for  selected  criterion 
measures  are  shown  in  Tables  8.7  -  8.18.  The  difference  between  these 
analyses  and  the  incremental  validities  presented  earlier  is  that  all  11 
variables  were  used  in  the  incremental  evaluation  and  the  procedure  was  not 
stepwise.  The  hierarchical  procedure  asked  for  the  optimal  6-variable 
equation . 

One  result  of  note  is  that  for  any  specific  criterion  measure  the  first 
four  variables  are  never  all  from  ASVAB  or  all  from  ABLE.  Another  general 
question  that  might  be  asked  is  whether  the  optimal  6-variable  equation 
provides  better  prediction  than  using  all  11  variables  each  time.  It  appears 
that  it  does  not  for  CTP  and  GSP,  but  does  provide  a  consistent  differential 
of  .04  -  ,06  for  ELS,  MPO,  and  PFB.  It  also  appears  that  ABLE,  most 
frequently  the  Dependability  scale,  does  play  a  role  in  predicting  CTP  and 
GSP.  This  contribution  is  masked  when  the  non-hierarch i ca 1  procedure  is  used. 


?05 


GO 

Q) 


csj 

o 

o 

o. 

on 


CJ 

on 


U. 

CSJ 

00 

1— 

CO 

z 

LJ 

CSJ 

on 

Q. 

u. 

o 

oo 

CJ 

CO 

on 

u. 

OJ 

CSJ 

oo 

CD 

QC 

oc 

3 

z 

CSJ 

on 

on 

CO 

u. 

u. 

oo 

oo 

on 

CD 

00 

on 

u. 

c\]  CSJ 

CVJ 

00 

o  o 

Ct 

CD 

O  OJ 

C_) 

CL  ^ 

Q. 

CSJ 

on  on 

on 

z 

U.  LU 

u. 

a. 

CO  oo 

oo 

o 

CD  OS 

CD 

on 

u. 

►—  k- 

OD 

z  z  z 

Z 

CO 

<r  «r  <c 

<r 

CD  o  rj 

CSI 

0-0  0 

cr 

z 

c  <  <c 

< 

s; 

o 

<c  <  «t 

< 

cv  CSJ  CSJ  og 

CSJ 

00 

z  z  z  z 

z 

CO 

a.  CL  CL  a. 

a. 

o  o  o  o 

o 

CSJ 

on  on  on  on 

on 

U.  u.  u. 

Li. 

QC 

00  00  00  CO 

00 

3 

CD  CD  CO  CO 

CD 

on 

3  n:  DC  31  ac 

z 

oo 

O  O  L-)  LJ  CJ 

o 

CO 

UJ  UJ  LU  UJ  UJ 

LU 

H"  ^  ^ 

Q 

«c  <  <  <c  «c 

< 

UJ 

^  ^ 

<  <  <  c  <c 

a. 

on 

< 

vD  Lf)  CVj  O 

vD 

Ln 

nn 

r>.. 

un  m 

CD 

^  <5*  ^  ^ 

<r 

a: 

•  •  *  ♦  • 

•  • 

UJ 

> 

c 

<c 

4/>  tn 

k— 

o  o 

z 

40 

< 

U  U 

03 

o  r-..  CO 

r>. 

•  ^  •  1— 

O’ 

O'  O  —1  CM  CSJ 

CSJ 

"O  no 

i/> 

—  Cvj  CVJ  CSI  fVj 

CSJ 

O)  OJ 

oj 

•  •  •  •  • 

■ 

L.  U 

4U 

D-  CL 

e 

z 

to  ^ 

o 

<4-J 

UJ 

</> 

• 

UJ 

< 

cclcxl 

u 

“O  -O 

<v 

0)  q; 

j 

40  w 

a> 

— 1  c\j  ro  ^  m 

'O 

1/1  t/* 

E 

3  3 

•— 5-^ 

u 

*o  "O 

'13 

<C  < 

Q. 

206 


Note.  N  -  477.  Unadjusted  and  corrected  for  range  restrictior.  <;ee  F-)'  a  8.2  for  names  of  variables  Raw 
construct. 
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Note.  N  =  413.  Unadjusted  and  corrected  for  range  restriction.  Raw  construct. 


Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE~114  Scores  Against  Effort/Leadership  (H8RAWELS):  Optimal 
Batteries  and  Weights  for  1-6  Predictors,  All  HOS _ 

Number  in 

^^odel  R-Square  R  Variables  in  Model 
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Note.  N  -  477.  Unadjusted  and  corrected  for  range  restriction.  Raw  construct. 


Table  8.10 

Multiple  Regression  of  4  ASVAB  factors  and  7  ABLE-114  Scores  Against  Effort/Leadership  Without  SJT 
(ELSNQSJT);  Optimal  Batteries  and  Weights  for  1-6  Predictors.  All  HOS _ 

Number  in 

Model  R-$quare  R  Variables  in  Model 
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Note.  N  -  477.  Unadjusted  and  corrected  for  range  restriction.  SJT  =  Situational  Judgment  Test.  Raw 
construct . 


Table  8.11 

Multiple  Regression  ot  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Persona]  Discipline  (H8RAWMPD): 
Optimal  Batteries  and  Weights  for  1-6  Predictors,  All  MOS _ 

Number  in 

Model  R-$quare  R  Variables  in  Model 
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Note.  N  -  477.  Unadjusted  and  corrected  for  range  restriction.  Raw  construct. 
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Note.  N  -  477.  Unadjusted  and  corrected  for  range  restriction.  Raw  construct. 
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Note.  N  =  477.  Unadjusted  and  corrected  for  range  restriction.  Raw  construct. 
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Note.  N  =  445.  Unadjusted  and  corrected  for  range  restriction. 


Q) 


vt 

O 

g- 

5 

u 

0) 

c 

o. 

u 

o 

a. 

-c 

V) 

u 

a» 

3 

>Q 

Q) 

•J 

U 

o 

«*• 

H- 

UJ  c/l 

o 

X 

VI 

c  ^ 

«Q  < 

<  •“ 

u 

4)  o 

u  ^ 

o  u 

o  .p- 

p"p 

go  ^ 

0.) 

T3 

^  %m 

O 

f-N  ^ 

X 

<  vO 

c 

UJ  1 

'  ••» 

3 

VI 

OJ 

n 

fM 

-Cl 

•^1 

3  VI 

*^l 

C  ^ 

V.I 

«  Z 

o> 

>1 

iA 

i~  4> 

O  2 

u  -o 

c 

u.  <9 

31 

3  io 

<  0) 

> 

</>  u 

<  0) 

4-» 

^  4-» 

a> 

CO 

u 

o 

3 

^  S 

O 

o  6 

vO 

1 

V)  ^ 

3 

v>  a. 

Of  o 

u 

O) 

Oi  •• 

lO  oc 

c 

o 

0)  a. 

®  ^  X 

V.  cu 

Q.VO 

<1>  T3 

<1;  ^ 

-o  o 

*—  •♦J  UJ 

e  s 

n  ^  CO 

<5  3  X 

Z 

<✓*) 

< 

C 
CNJ  CM 

z  z 

Q.  O. 
Q  O 

U.  Lk. 
00  00 
00  00 

CNJ  C\j  Osj 

Z  2  Z 
31  ^  ^ 
O  Q  O 
OO  (-0  <j0 
U.  lu 

00  CO  00 
CO  CO  CO 


Z  Z  2  Z 
«c  <c:  <  < 
0  3  2  3 

CrcrOO 

<  <  <c 

<  <  <t  «c 

CM  <\l  CM  CM  CVJ 
^  ^  ^  ^  ^ 
Qc  q:  a  q:  q: 

3:  :&  :e  2  :s 

00  VO  CfO 

U»  W  .  u.  u. 
00  OO  OO  00  00 
00  CD  3  3  3 


fM  rx.  C7>  0^  O 
O  “O  <T>  o 

rn  no  m  ^ 


^  CM  •— <  cyv  O 

g  ^  lo  ir>  o 


CM  no  ^  OO 


CM 

O. 

o 

o 

CO 

Iw 

‘30 

3 


LU 

CL 

VO 

< 


CM 

2 

Q 

3 

3 

CM 

2 

X 

3 

CO 

u. 

3 

m 


2 

*X. 

3 

O 


CM 

3k: 

QC 


00 

3 


3 


3 


CM 

1 

1 

1 

1 

1 

1 

1 

1 

4 

1 

t 

1 

1 

1 

1 

1 

1 

1 

4 

1 

O 

1 

1 

1 

1 

3> 

1 

t 

1 

1 

3 

1 

1 

1 

1 

IVO 

1 

1 

1 

1 

U. 

• 

• 

eo 

< 

1 

1 

1 

3 

1 

1 

1 

1 

CM 

1 

1 

1 

1 

i 

1 

4 

1 

h— 

1 

1 

1 

1 

Z 

1 

4 

1 

1 

LJ 

1 

1 

1 

1 

VO 

1 

1 

1 

1 

3 

• 

• 

3 

1 

1 

1 

1 

3 

I 

4 

1 

1 

CM 

1 

1 

t 

1 

1 

1 

1 

1 

3 

1 

1 

1 

1 

o 

1 

1 

1 

1 

1 

t 

4 

1  no 

uO 

1 

1 

1 

«  O 

3 

• 

1  • 

• 

3 

I 

1 

1 

1 

3 

I 

1 

4 

1 

CM 

1 

1 

1 

1 

1 

1 

1 

1 

3 

1 

1 

1 

1 

VO 

t 

1 

1 

1 

1 

1 

1 

UO 

1 

1 

1 

1 

3 

• 

• 

3 

1 

1 

1 

i 

1 

1 

1 

1 

CM 

1 

• 

1 

1 

1 

1 

j 

1 

Z 

1 

1 

1 

1 

3 

i 

»  no 

1  no 

i  o 

O 

1 

t  o> 

»  3 

1 

t/0 

1 

'  O 

1  o 

1  O 

3 

« 

1  • 

• 

3 

1 

1 

1 

1 

3 

t 

1 

1 

1 

CM 

1 

I 

1 

i 

1 

1 

1 

1 

Z 

4 

4 

1 

1 

X 

t 

ro 

1  o^ 

1  3 

1 

o 

1 

CM 

»  CM 

1  CM 

1  CM- 

vO 

1 

M4 

1  mm 

3 

• 

1  • 

1  ^ 

1  , 

3 

1 

1 

1 

4 

CD 

1 

1 

1 

t 

CM 

t 

1 

1 

1 

1 

4 

1 

1 

1 

1 

1 

t 

3 

vO 

1  o 

CO 

1  m 

1  ^ 

4  no 

2 

o 

1  3 

3 

»  3 

1  m 

•  ^ 

VO 

ro 

i  CM 

1 

1  mm 

3 

• 

• 

1  • 

3 

1 

1 

1 

4 

3 

t 

1 

1 

1 

O 

1 

> 

1 

1 

1 

4 

1 

3 

1 

I 

1  UO 

1  nj 

3 

» 

1 

•  I/O 

•  3 

3 

1 

1 

»  o 

1  o 

vO 

• 

. 

C 

J 

1 

1 

1 

OO  CM 

1 

1 

1 

1 

OO 

c 

1 

1 

1 

1 

no  no 

1 

1 

1 

1 

•  • 

3 

1 

1 

1 

1 

3 

1 

1 

1 

1 

3 

1 

1 

t 

4 

3 

1 

1 

1 

1 

CO  SO 

>• 

• 

, 

«c 

i 

1 

1 

1 

o  o 

1 

1 

1 

1 

c 

1 

1 

1 

1 

u  u 

1 

1 

1 

1 

1 

1 

1 

3  T3 

so 

•  «r 

no 

•  n^ 

•  o 

1  1 

0;  a> 

o 

< 

1  CM 

•  O 

1 

1 

g.  w 

3) 

1  CM 

CM 

•  CM 

3  3 

• 

1  , 

1  ^ 

£ 

< 

1 

1 

1 

1 

cO 

t 

1 

1 

1 

< 

4 

I 

t 

1 

CO 

1 

1 

1 

1 

»  - 

UJ 

3 

1 

i 

1 

1 

3|3| 

3 

1 

1 

1 

i 

w 

3 

1 

1 

1 

1 

3  T3 

<l> 

►— 

• 

<u  o; 

«•» 

< 

I 

1 

1 

■4~J  «_) 

<D 

1 

1 

1 

SO  i/> 

E 

<£ 

1 

1 

1 

1 

3  3 

1 

1 

u 

t 

3  -O 

1 

c  <x. 

3 

— H 

1  CM 

1 

OO 

i  ^ 

1 

t  UO 

1 

1  O 

1 

<L> 

O 


214 


509.  Unadjusted  and  corrected  for  range  restriction. 


Tabu  8.16 

Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Criterion  Factor  1*  (CRiTCGTl): 
Optimal  Batteries  and  Weights  for  1-6  Predictors.  All  HOS 
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Note.  N  =  413.  Unadjusted  and  corrected  for  range  restriction. 
Core  Fechnical  +  General  Soldiering  +  Tra in ing/Counsel ing. 


Note.  N  =  477.  Unadjusted  and  corrected  for  range  restriction. 
*  Effort/Leadership  +  Personal  Discipline  +  Physical  Fitness. 


Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Hands-On  Average  (M8XHT0TT):  Optimal 
Batteries  and  Weights  f-r  1-6  Predictors,  All  HOS 
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Note.  N  -  477.  Unadjusted  and  corrected  for  range  restriction. 


A  summary  description  of  the  six  best  predictors  in  hierarchical  order 
for  each  of  eight  criterion  variables  is  given  in  Table  8.19.  Selecting 
predictors  with  the  corrected  versus  uncorrected  covariance  matrix  makes 
virtually  no  difference  in  the  predictor  battery  selected. 


Genera lizability 

The  hierarchical  optimal  battery  analyses  for  selected  criterion  scores 
within  the  three  MOS  clusters  are  shown  in  Tables  G.20  -  8.37.  Although  a 
more  formal  test  is  still  required,  the  content  of  the  optimal  equations  does 
seem  to  be  different  across  clusters  for  Core  Technical  Proficiency  and 
perhaps  also  for  General  Soldiering  Proficiency  and  Effort/Leadership,  but  not 
for  Personal  Discipline,  Physical  Fitness,  and  Training/Counseling. 

A  descriptive  picture  of  the  general izabi  1  ity  of  prediction  equations 
across  performance  factors  (for  the  combined  sample)  and  across  MOS  clusters 
for  selected  performance  factors  is  shown  in  Tables  8.38  -  8.39.  All  entries 
are  multiple  correlations  and  the  diagonals  represent  estimates  based  on 
optimal  weights.  Estimates  of  what  happens  when  less  than  optimal  weights  are 
used  to  predict  that  same  criterion  are  obtained  by  looking  across  the  rows. 
Estimates  of  what  happens  when  a  particular  set  of  weights  is  applied  to  other 
criterion  measures  or  other  MOS  are  obtained  by  looking  down  the  columns. 

All  estimates  are  based  on  the  corrected  covariance  matrix.  The 
diagonals  are  adjusted  for  shrinkage  using  the  Rozeboom  formula  with  k  =  6. 

The  off-diagonals  are  not  adjusted  because  the  weights  were  not  computed 
against  that  particular  dependent  variable.  The  values  in  the  diagonals  of 
Tables  8.36  and  8.39  will  vary  slightly  from  the  corresponding  estimates  in 
Tables  8.7  -  8.37  because  all  estimates  in  Tables  8.38  and  8.39  were  based  on 
exactly  the  same  sample.  The  samples  in  Tables  8.7  -  8.37  varied  slightly  by 
criterion  variable  or  by  MOS  group. 

As  shown  in  Table  8.38,  within  MOS  clusters  there  is  very  little 
differential  validity  for  CTP  vs.  GSP.  Either  set  of  weights  works  about  as 
well.  However,  the  same  is  net  the  case  for  the  other  four  performance 
factors.  Better  prediction  is  always  achieved  by  using  the  equation  developed 
for  each  factor. 

As  portrayed  in  Table  8.39,  the  greatest  degree  of  differential  validity 
across  MOS  groups  is  for  GSP  and  ICS,  not  CTP.  However,  the  sample  sizes  for 
group  2  and  group  3  are  not  large  and  firm  conclusions  will  be  dependent  on  an 
appropriate  examination  of  sampling  error,  which  is  yet  to  be  done. 

A  summary  of  the  mean  diagonal  entries  versus  the  mcen  off-diagonal 
entries  in  Tables  8.38  and  8.39  is  shown  in  Tables  8.40  and  0.41.  For 
example.  Table  8.41  shows  that  if  the  weights  for  the  optimal  battery  in  one 
cluster  are  applied  to  those  same  variables  in  the  other  two  clusters,  the 
level  of  validity  is  much  lower.  The  greatest  difference  in  validity  across 
MOS  clusters  is  for  GSP,  while  the  smallest  difference  is  for  ELS. 
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Note.  N  -  218.  Unadjusted  and  corrected  for  range  restriction. 


Table  8.21 

Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-llA  Scores  Against  General  Soldiering  Proficiency  (M8RAWGSP): 
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Note.  N  -  218.  Unadjusted  and  corrected  for  range  restriction. 
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Note.  N  ■  218.  Unadjusted  and  corrected  for  range  restriction. 
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Table  8.26 

Multiple  Regression  of  4  ASVAB  Factors  and  7  AfiLE-114  Scores  Against  Core  Technical  Proficiency  (H8RAWCTP): 
Opt iroal  Batteries  and  Weights  for  1-6  Predictors,  Technical  HOS* _ 
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Noie.  N  =  95.  Unadjusted  and  corrected  for  range  restriction. 
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Note.  N  =  96.  Unadjusted  and  corrected  or  range  restriction. 
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Note.  N  =  96.  Unadjusted  and  corrected  for  range  restriction. 
•  MOS  Group  2  (31C,  63B). 


Nultiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Personal  Discipline  (HBRAWrlPO):  Optinal 
Batteries  and  Weights  for  1-6  Predictors,  Technical  MOS* 
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Note.  N  96.  ’inadjusted  and  corrected  for  range  restriction. 
“  MOS  Group  2  (31C,  63B). 
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Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Training/Counseling  (H8RAWTCS):  Optimal 
Batteries  and  Weights  for  1-6  Predictors,  Technical  M0S‘ _ 
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Note.  N  -  96.  Unadjusted  and  corrected  for  range  restriction. 
•  MOS  Group  2  (31C.  63B). 


Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Core  Technical  Proficiency  (H8RAWCTP); 

Optimal  Batteries  and  Weights  for  1-6  Predictors,  Support  HOS* 
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Note.  N  =  III.  Unadjusted  and  corrected  for  range  restriction. 
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Note.  N  -  111.  Unadjusted  and  corrected  for  range  restriction. 
•  MOS  Group  3  (71L,  91A). 


Table  8.34 

Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Effort/Leadership  (M8RAWELS):  Optlnuil 
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Note.  N  =  111.  Unadjusted  and  corrected  for  range  restriction. 
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Note.  N  -  111.  Unadjusted  and  corrected  for  range  restriction. 

A  ^  J  '—a  ^  ^  m.  .1  ^ 


Table  8.3S 

Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Physical  Fitness  (M8RAWPFB):  Optimal 
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Note.  N  =  111.  Unadjusted  and  corrected  for  range  restriction 

a  » Art  r-rt..  ^ 


Table  8.37 

Multiple  Regression  of  4  ASVAB  Factors  and  7  ABLE-114  Scores  Against  Training/Counseling  (MSRAVfTCS):  Optimal 
Batteries  and  Weights  for  1-6  Predictors.  Support  MOS* _ _ _ 

Number  in 

Model  R-$quare  R  Variables  in  Model 
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Note.  N  =  111.  Unadjusted  and  corrected  for  range  restriction. 
*  MOS  Group  3  (71L.  91A). 


Rcms  are  criteria:  colians  are  weights  corrected  for  range  restriction;  oultiple  R  for  optiaal  weights  in  tx>ld:  Roretxxa  adjustments  in  parentheses, 
iterion  factor  I  •  CfP*CSP«ICS.  ~ 

iter  ion  factor  2  ' 


Table  8.39 


Multiple  Correlations  for  Three  Sets  of  Weights  and  Eight  Criteria  Optima! 
Weiahts  for  Each  HOS  Group  Applied  to  All  Others 


Note.  Rows  are  MOS  group;  columns  are  weights  corrected  for  range 
restriction;  multiple  R  for  optimal  weights  in  bold;  Rozeboom  adjustments  in 
parentheses.  MOS  Group  1  =  Combat  specialties  (IIB,  13B,  19R,  95B/N);  MOS 
Group  2  =  Technical  specialties  (31C,  63  B/N);  Group  3  =  Support  specialties 
(71L,  91  A/N). 


(Continued) 
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Table  8.39  (Continued) 


Multiple  Correlations  for  Three  Sets  of  Weights  and  Eight  Criteria  Optimal 
Weights  for  Each  HOS  Group  Applied  to  All  Others  _ 


Physical  Fitness  (M8RAWPFB) 


OSl  MOS2  M0S3 
ghts  Weights  Weights 


.154  .182 


.362 


Training/Counseling  (M8RAWTCS) 


MOSl  MOS2  M0S3 
Weights  Weights  Weights 


MOSl 


M0S2  .177 


M0S3  .008 


Core  Tech/Gen  Soldier  (M8CTPGSP) 


OSl  M0S2  M0S3 
ghts  Weights  Weight 


MOSl 


M0S2  .454 


.236  .217 


.446 


.257  .336 


Effort-Lead/Discipline  (M8ELSMPD) 


MOSl 


MOS2 


M0S3 


.488 

(.445) 

.246 

.335 

.504 

(.397) 

.314 

.231 

Note.  Rows  are  MOS  group;  columns  are  weights  corrected  for  range 
restriction;  multiple  R  for  optimal  weights  in  bold;  Rozeboom  adjustments  in 
parentheses.  MOS  Group  1  =  Combat  specialties  (IIB,  13B,  19R,  95B/N);  MOS 
Group  2  =  Technical  specialties  (31C,  63  8/N);  Group  3  =  Support  specialties 
(71L,  91  A/N). 
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Table  8.40 

Multiple  Correlations  Using  Optitoal  Weights  and  Mean  Multiple  Correlations 
When  Weights  From  the  Other  Criteria  Are  Used  for  Six  Criterion  Measures 


Criterion 

Multiple  R* 
Using 

Optimal  Weights 

Mean  R  When 

Using  Weights  From 
the  Other  Criter  ia 

Core  Technical  (M8RAWCTP) 

.429 

.264 

General  Soldiering  (M8RAWGSP) 

.557 

.334 

Effort/Leadership  (M8RAWELS) 

.482 

.303 

Personal  Discipline  (M8RAWMPD) 

.169 

.102 

Physical  Fitness  (M8RAWPFB) 

.375 

.105 

Training/Counseling  (M8RAWTCS) 

.231 

.13 

Mean  Over  Six  Criteria 

.374 

.206 

^  Adjusted  by  Rozeboom. 


Table  8.41 


Mean  Multiple  Correlations  for  Eight  Criteria  Computed  Across  Three  HOS  Groups 
When  Optimal  Weights  Are  Used^  and  When  Weights  Obtained  From  the  Other  MOS 
Groups  Are  Used _ 


Criterion 

Mean  Adjusted^ 

R  With 

Optimal  Weights 

Mean  R  Using  vie^ghts 
From  the  Other 

MOS  Groups 

Core  Technical  (M8RAKCTP) 

.423 

.375 

General  Soldiering  (M8RAWGSP) 

.409 

.161 

Effort/Leadership  (M8RAWELS) 

.486 

.403 

Personal  Discipline  (M8RAHMP0) 

.222 

.087 

Physical  Fitness  (M8RAWPFB) 

.404 

.267 

Training/Counseling  (M8RAWTCS) 

.283 

.079 

CTP+GSP  (M8CTPGSP) 

.466 

.324 

ELS+MPD  (M8ELSMP0) 

.368 

.306 

^  Adjusted  by  Rozeboom. 
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SUMMARY  CONCLUSIONS 


In  general,  in  spite  of  the  small  samples  for  each  MOS  and  the  necessity 
of  regarding  all  mean  criterion  differences  as  error  (i.e.,  standardizing 
criterion  scores  within  MOS),  the  validities  for  ASVAB  and  ABLE  were  as  high, 
or  higher,  for  predicting  second-tour  performance  as  for  predicting  first-tour 
performance.  Unit  weights  did  not  weaken  the  validities  for  ASVAB  but  they 
did  constrain  the  predictor  accuracy  of  ABLE. 

A  consistent  finding  from  the  hierarchical  analysis  is  that  for  Core 
Technical  Proficiency,  General  Soldiering  Proficiency,  and  Effort/Leadership 
criteria,  the  optimal  predictor  battery  is  never  composed  of  only  ASVAB  or 
only  ABLE  factor  scores.  For  example,  the  Dependability  factor  from  the  ABLE 
is  a  constant  predictor  of  the  "can  do"  component  of  performance. 

Finally,  based  on  the  above  analyses,  there  appears  to  be  more 
differential  validity  across  MOS  for  the  second  tour  samples  than  was  found 
during  the  analyses  of  the  first-tour  data  in  CVI. 

All  of  these  issues  can  be  analyzed  more  rigorously  when  the  larger 
samples  and  fuller  set  of  predictor  measures  become  available  as  a  result  of 
the  second-tour  longitudinal  (LVII)  validation. 
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Chapter  9 

PREDICTION  OF  SECOND-TOUR  PERFORMANCE  FROM  THE  TRIAL  BATTERY 
AND  FROM  FIRST-TOUR  PERFORMANCE 

Norman  6.  Peterson  and  Rodney  L.  Rosse 

The  original  research  designs  for  Project  A  ano  Career  Force  include  the 
concept  of  combining  successive  pieces  of  information  from  (a)  predictor  tests 
administered  at  entry,  (b)  measures  of  performance  during  training,  and  (c) 
measures  of  first-tour  job  performance  to  predict  individual  performance  in 
the  second  tour  of  duty.  Such  a  "roll-up"  model  involves  incremental 
augmentation  of  predictor  information  as  soldiers  move  through  their  careers. 
Specifically,  entry  test  scores  may  be  combined  with  known  actual  performance 
at  any  one  point  so  as  to  make  more  complete  use  of  available  information  for 
prediction  of  performance  in  the  second  or  later  tour  of  duty. 

The  k,ol  lection  of  second-tour  performance  data  (CVII  data)  for  at  least 
some  members  of  the  Concurrent  Validation  sample  (CVI)  presented  the  first 
opportunity  to  examine  this  concept  with  empirical  data.  Test  scores  (from 
ASVAB  and  experimental  tests)  were  expected  to  predict  performance  in 
training.  In  turn,  the  test  scores  and  training  performance  should  predict 
job  performance  during  the  first  tour  of  duty.  And  final^v,  the  test  scores, 
training  performance,  and  first-tour  job  performance  should  predict  second- 
tour  job  performance. 

This  chapter  describes  analyses  examining  the  relationship  of  ASVAB 
scores  (given  at  the  time  recruits  entered  the  Army),  the  CVI  predictor  scores 
(i.e.,  the  Project  A  CVI  Trial  Battery,  the  preliminary  version  of  the 
Experimental  Predictor  Battery,  given  during  the  first  tour),  and  first-tour 
job  performance  scores  with  second-tour  job  performance  scores.  However, 
there  were  some  complications  with  the  analyses.  Available  sample  sizes  for 
this  preliminary  exploration  were  extremely  small,  and  it  was  unclear  exactly 
how  to  account  for  range  restriction  for  a  sample  of  this  type.  The  approach 
to  these  issues  is  described  below. 


THE  SAMPLE 

There  were  121  soldiers  m  Batch  .A  MOS  who  had  been  assessea  on  at  least 
a  subset  of  measures  during  the  CVI  ana  CVII  data  collections.  Not  all  121 
soldiers  had  complete  CVI  and  CVII  data.  The  minimum  number  of  soldiers  for  a 
given  combination  of  CVI  and  CVII  measures  was  102.  Table  9.1  shows  the 
maximum  number  of  soldiers  who  had  CVI  and  CVII  data,  by  MOS. 


MEASURES 

Criteria 

The  second-tour  perfo''mance  criterion  CVII  measures  used  in  the  analysis 
were  the  raw  and  residual  score?  for  the  five  constructs  first  identified 
during  the  first-tour  Concurrent  Validation,  and  confirmed  by  the  CVII 
modeling  analysis  (see  Chapter  8,  "Results  the  Socona-"our  Validation 
(evil),"  for  more  detail  on  the  development  of  the  criterion  scores  for 
second-tour  performance).  These  five  constructs  are  called  Core  Technical 


Proficiency,  General  Soldiering  Proficiency,  Effort  and  Leadership, 
Maintaining  Personal  Discipline,  and  Physical  Fitness  and  Military  Bearing. 

Table  9.1 

Numbers  of  Soldiers  With  CVI  and  CVII  Data  by  HOS _ 

MOS  N  _ 


IIB 

Infantryman 

8 

13B 

Cannon  Crewman 

26 

19E 

M60  Armor  Crewman 

4 

31C 

Single  Channel  Radio  Operator 

8 

63B 

Light-Wheel  Vehicle  Mechanic 

25 

64C 

Motor  Transport  Operator 

7 

71L 

Administrative  Specialist 

15 

91A 

Medical  Specialist 

15 

95A 

Military  Police 

13 

Total 

121 

Predictors 

Predictor  measures  came  from  the  ASVAB,  from  the  Project  A  CVI  Trial 
Battery,  and  from  first-tour  job  performance  measures.  We  used  the  least- 
squares  weights  developed  for  the  CVI  criterion  constructs  rather  than 
developing  new  weights  for  CVII  criterion  constructs  because  of  the  extremely 
limited  sample  sizes.  There  were  four  ASVAB  factor  composites,  six  composites 
of  computer-administered  test  scores,  one  spatial  ability  composite,  six 
vocational  interest  (AVOICE)  composites,  four  temperament/biodata  (ABLE) 
composites,  and  three  composites  from  the  Job  Orientation  Questionnaire  (JOB). 
These  measures  and  their  validities  for  the  CVI  sample  are  fully  described  in 
the  Summer,  1991  issue  of  Personnel  Psychology  (McHenry  et  al.,  1990,  Peterson 
et  al . ,  1990) . 

ft 

The  soldiers'  scores  on  these  ASVAB  and  Trial  Battery  composites  were 
combined  by  using  the  least-squares  weights  developed  during  CVI  analyses  for 
each  criterion  construct  for  each  MOS.  For  example,  there  exists  a  set  of 
least-squares  weights  developed  during  CVI  for  the  four  ASVAB  factor  composite 
scores  for  predicting  Core  Technical  Proficiency  for  each  MOS.  These  weights 
were  applied  to  the  soldiers'  ASVAB  scores  to  obtain  an  "ASVAB-Core  Technical" 
predicted  score.  This  predicted  score  could  then  be  correlated  with  the 
soldiers'  Core  Technical  Proficiency  score  for  CVII. 

Predicted  scores  were  computed  for  each  battery/criterion  combination 
separately,  that  is,  for  ASVAB/Core  Technical  Proficiency,  ASVAB/General 
Soldiering  Proficiency,  ABLE/Core  Technical  Proficiency,  ABLE/Physical  Fitness 
and  Military  Bearing,  and  the  other  combinations.  Also  computed  were 
predicted  scores  for  ASVAB  plus  each  of  the  parts  of  the  Trial  Battery  (e.g., 
ASVAB  +  ABLE  weighted  to  predict  Core  Technical,  ASVAB  +  computer-administered 
scores  weighted  to  predict  Core  Technical). 
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The  raw  and  residual  scores  achieved  by  the  soldiers  on  the  five 
criterion  constructs  during  the  CVI  data  collection,  or  the  "first-tour" 
criterion  scores,  were  also  used  as  predictors  of  second-tour  job  performance. 


PROCEDURE 

CVI  predictor  scores  were  correlated  with  the  CVII  criterion  scores  in 
two  ways:  (a)  correlations  were  computed  within  each  MOS  and  these  values 
were  averaged  (weighted  by  N),  and  (b)  correlations  were  computed  across  the 
total  sample.  Correlations  with  CVII  criteria  were  computed  separately  for 
the  ASVAB,  Spatial,  Computer-administered,  ABLE,  AVOICE,  and  JOB  composites 
and  for  the  CVI  criterion  scores.  Correlations  were  also  computed  for  the 
ASVAB  plus  each  of  the  other  predictor  sets  from  the  Trial  Battery  and  the  CVI 
criteria.  When  the  CVI  criteria  were  combined  with  any  of  the  other  predictor 
scores,  they  were  standardized  within  MOS  (using  the  larger  CVI  samples  to 
compute  standard  scores)  and  summed  to  achieve  equal  weighting  between 
ASVAB/Trial  Battery  and  CVI  criterion  scores. 

There  was  ample  opportunity  for  restriction  of  range  on  all  of  the 
predictor  measures.  The  CVII  sample  is  a  subset  of  soldiers  who  were 
originally  selected  for  entry  into  the  Army  on  the  basis  of  their  ASVAB 
scores.  Additional  range  restriction  resulted  from  attrition  during  Basic  and 
Advanced  Individual  training.  More  personnel  attrited  after  training  but 
before  the  end  of  their  first  tour,  for  a  variety  of  reasons.  Finally,  many 
had  chosen  not  to  reenlist  for  a  second  tour  and  others  had  not  been  qualified 
to  reenlist. 

Because  of  the  number  of  different  points  at  which  additional  range 
restriction  could  occur,  there  are  a  number  of  different  "populations"  to  which 
the  CVII  sample  could  be  corrected.  If  the  problem  is  to  select  second-tour 
soldiers  from  experienced  first-tour  personnel,  then  the  set  of  all  persons  who 
are  nearing  completion  of  the  first  tour  seems  the  most  appropriate  population. 
We  decided  to  consider  the  latter  group  as  the  population  and  to  use  the  full 
CVI  sample  statistics  as  the  estimates  of  the  "population"  parameters  for 
making  range  restriction  corrections.  All  corrections  were  made  using  the 
formula  outlined  in  Lord  &  Novick  (1968,  p.  147). 


RESULTS 

The  correlations  of  scores  on  the  first-tour  criteria  with  scores  on 
second-tour  criteria  are  shown  in  Tables  9.2  and  9.3  for  the  raw  and  residual 
criteria,  respectively.  These  tables  show  the  weighted  means  of  correlations 
computed  within  MOS.  The  correlations  are  not  corrected  for  restriction  of 
range.  The  note  for  each  table  shows  the  mean  of  the  diagonal  correlations, 
which  contains  the  correlations  of  the  same  criteria  across  first  and  second 
tour--that  is,  the  correlation  of  Core  Technical  between  first  and  second 
tour,  the  correlation  of  General  Soldiering  between  first  and  second  tour,  and 
so  on.  This  mean  is  an  index  of  convergent  validity  for  the  set  of  criterion 
constructs.  The  note  also  shows  the  mean  of  the  off-diagonal  correlations-- 
that  is,  the  correlations  between  different  criterion  constructs  across  first 
and  second  tour.  The  difference  between  the  mean  diagonal  and  mean  off- 
diagonal  correlation  can  be  thought  of  as  an  indicator  of  discriminant 
validity. 
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The  convergent  validities  in  the  two  tables  range  from  .16  to  .50  and 
are  a  bit  higher  for  the  raw  criteria  than  for  the  residual  criteria  (.33 
versus  .28,  averaged).  Both  convergent  validities  are  higher  than  the  off- 
diagonal  validities  by  about  the  same  margin:  .18  for  the  raw  criteria  and  .16 
for  the  residual  criteria.  Considering  the  small  samples,  the  size  and 
pattern  of  these  correlations  seem  impressive. 

Table  9.2 


Mean  Uncorrected  Correlations  of  CVI  Raw  Criterion  Composites  with  CVII  Raw 
C r iterion  Composites:  Computed  Within  HOS  and  Averaged _ 


CVI 

CVII 

Criterion 

Composite 

Composite 

CTP 

GSP 

ELS 

MPD 

PF8 

Core  Technical 

Prof  iciency 

.37 

.43 

.19 

.04 

.05 

General  Soldiering 
Prof i c  iency 

.45 

.40 

31 

.07 

.13 

Effort  and 

Leadershi p 

.23 

.OQ 

.26 

.20 

.08 

Maintaining  Personal 
Discipline 

.13 

.21 

.07 

.16 

.08 

Physical  r  'tness  and 
Mi  1 i tary  Bearing 

.04 

.03 

.15 

.22 

^8 

fio'.e:  Hs  ■  102  -  121.  Mean 

diagonal  value 

-  .23:  (Tiean  off 

•diagonal  value  •  .15. 

Table  9.3 

Mean  Uncorrected  Correlations  of 
Residual  Criterion  Composites: 

evT  Residual  Criterion  Composites 
Computed  Within  MOS  and  Averaged 

with  CVII 

CVII 

Cri terion 

Composite 

CVI  Criterion 
Composite 

CT? 

GSP 

ELS 

MPD 

PFB 

Core  Technical 

Prof iciency 

.32 

.13 

.03 

-.03 

General  Soldiering 
Prof i c  iency 

.24 

.24 

.06 

.03 

Effort  and 

Leadership 

.08 

.17 

.12 

.20 

Maintaining  Personal 
Disc ipl  ine 

.01 

.24 

.01 

.17 

.17 

I'hysical  '-i  tness  and 
Ml ■ 1 tary  Lear  'ng 

-.04 

.10 

.14 

.18 

feat:  II.  •  ■  :?),  ML'd'i 

.aiui- 

-  ?li.  ■’•.•dl-  uff 

•  diajoii.il  -.aijc  •  12. 
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Tables  9.4  and  9.5  show  the  same  results  as  Tables  9.2  and  9.3  except 
that  the  correlations  were  computed  across  the  total  sample  rather  than 
compute-  within  MOS  and  averaged.  The  result:  are  very  similar.  The 
convergent  validities  are  a  bit  higher  (.06  'jreater  average  validity  for  the 
raw  criteria  and  .04  greater  for  the  residual  criteria),  but  the  off-diagonal 
validities  are  within  one  or  two  points  and  the  pattern  is  very  similar. 

Table  9.4 


Uncoirected  Correlations  Between  CVI  Raw  Criterion  Composites  and  CVII  Raw 
Criterion  Composites:  Computed  Across  Total  Sample 


CVII 

Criterion 

Composite 

CVI  Criterion 

Compos i te 

CTP 

6SP 

ELS 

MPO 

PFB 

Core  Technical 

Prof ic iency 

.47 

.48 

.22 

.10 

.08 

General  Soldiering 

Prof ic iency 

.47 

.36 

.13 

.17 

Effort  and 

Leadersh ip 

.19 

.07 

■  30 

.19 

.13 

Maintaining  Personal 
Disc  . ■ ine 

.06 

.14 

.16 

.26 

.19 

Physical  Fitness  and 

Mi  1  itary  Bearing 

.00 

-.04 

.15 

.15 

.48 

Note;  N  •  102  •  121.  Mean  diagonal  valje  •  .39;  mean  oft -diagonal  value  •  .1'. 


t  able  9 . 5 

Uncorrccted  Correlations  Between  CVI  Residual  Criterion  Composites  and  CVII 
Residual  Criterion  Composites:  Computed  Across  Total  Sample _ 

CVli  Criterion  Composite 


CVI  Criterion 


Compos i te 

CTP 

GSP 

El.S 

MPO 

PFB 

Core  Technical 

Prof ic iency 

.35 

.34 

.12 

.03 

-.01 

General  Soldiering 

Prof i : iency 

.24 

■  20 

.24 

.07 

.10 

Effort*  and 

Leadership 

,13 

.16 

.30 

.13 

.26 

Ma  inta i n i no  Persona  1 

D  1  sc  i  ()  1  i  ne 

-.03 

.17 

.06 

■  21 

.22 

Phys ica 1  F itness  and 
Military  Bear  i  rig 

1 

1  t 

1  o 

.04 

.15 

.11 

■  55 

N'jtL*:  N  •  10?  -  121.  Meal'  jijgofial  value  •  .3?;  of  f  •  d  .ag-iria  I  value  •  .13. 
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Tables  9.6  and  9.7  show  the  cori'elations  with  CVII  criteria  (raw  and 
residual  scores)  of  predicted  scores  based  on  CVI  weights  for  ASVAB  and  Trial 
Battery  composites,  as  well  as  for  CVI  criterion  scores.  The  values  shown  are 
weighted  means  of  correlations  computed  within  MOS.  For  example,  the  first 
cell  in  the  tables  shows  the  mean  correlation  with  scores  on  CVII  Core 
Technical  Proficiency  of  the  predicted  scores  computed  by  applying  the  least- 
squares  weights  for  ASVAB  factors  for  predicting  Core  Technical  Proficiency  in 
CVI.  The  correlation  in  parentheses  is  corrected  for  range  restriction.  The 
values  in  the  rest  of  the  first  line  show  similar  correlations  for  the  other 
criteria.  The  values  in  the  second  line  of  the  table,  labeled  CVI,  reproduce 
the  diagonal  or  convergent  validities  from  Tables  9.2  and  9.3,  as  well  as 
showing  the  values  corrected  for  range  restriction.  The  rest  of  the  table 
shows  the  correlations  with  CVII  criteria  of  combinations  of  the  ASVAB,  Trial 
Battery  composites,  and  CVI  criteria. 

Examination  of  these  tables  shows  that,  on  the  whole,  the  CVI  criterion 
scores  have  the  highest  correlations  with  CVII  criterion  scores.  The  Spatial 
composite  has  slightly  higher  validities  for  Core  Technical  and  General 
Soldiering  Proficiency  (the  two  "can  do"  criteria)  and  the  JOB  composite  has  a 
bit  higher  validity  for  Maintaining  Personal  Discip^'ne,  but  the  CVI  criterion 
scores  rank  first  or  second  on  all  five  measure's.  ding  the  ASVAB  and  the 
ASVAB  plus  Trial  Battery  composite  scores  to  i.l  scores  does  increment  the  CVI 
validity  coefficients  for  Core  Technical  and  General  Soldiering. 

The  ASVAB  validities  follow  the  familiar  pattern  of  predicting  the  two 
"can  do"  criteria,  but  not  predicting  the  "will  do"  criteria  very  well.  The 
Spatial  and  Computer  composites  also  follow  this  pattern.  The  Spatial 
composite  validities  on  the  "can  do"  criteria  are  even  higher  than  those  for 
the  ASVAB. 

As  noted,  the  JOB  unexpectedly  did  the  best  job  of  predicting 
Maintaining  Personal  Discipline  and  also  predicted  Effort  and  Leadership 
better  than  anything  except  CVI  performance.  The  ABLE  correlations  are  lower 
than  expected  for  the  "will  do"  criteria.  The  AVOICE  correlations  are 
generally  low,  but  are  somewhat  greater  'or  the  "can  do"  than  for  the  "will 
do"  criteria.  The  correlation  between  the  CVI  and  CVII  measures  of  Physical 
Fitness  and  Military  Bearing  is  almost  twice  as  great  as  the  correlation 
between  the  ABLE  composite  and  this  criterion. 

The  validities  for  the  raw  scores  on  the  CVII  criteria  (Table  9.6)  are 
generally  higher  than  the  validities  for  the  residual  criteria  (Table  9.7). 
Also,  it  appears  that  correcting  for  range  restriction  had  little  effect  on 
the  results.  In  addition,  the  within-MOS  range  restriction  corrections  had 
shown  little  effect . 

Table  9.8  and  9.9  show  the  same  data  as  Tables  9.6  and  9.7  except  that 
the  correlations  are  computed  across  the  total  sample  and  no  correlations  are 
shown  that  have  been  corrected  for  range  restriction,  (Because  of  important 
differences  between  the  MOS  in  terms  of  criterion  construct  scores,  we  did  not 
think  It  appropriate  to  perform  range  restriction  corrections  across  all  MOS.) 
The  pattern  of  correlations  appears  to  be  very  similar  to  those  shown  in 
Tables  9.6  and  9.7,  but  the  values  are  generally  higher.  This  may  be  due  to 
the  inclusion  of  between-MOS  differences  in  the  calculation  of  the 
corre '  a  t  ions  across  she  total  sample,  which,  could  operate  to  increase  the 
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Table  9.7 

Uncorrected  and  Corrected  Correlations  Between  CVI-Weighted  Predictor  Composites,  CVI  Criterion  Composites, 

and  evil  Criterion  Composites  for  Residual  Scores:  Computed  Within  HOS  and  Averaged _ 
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erion  Scores  and  Predictor  CuBposites  were  standardited  within  MOS  and 


Table  9.8 


Correlations  Between  CVI-Weighted  Predictor  Coraposites,  CVI  Criterion 
Composites,  and  CVII  Criterion  Composites  for  Raw  Scores:  Computed  on  Total 
Sample _  _ 


Predictor  and  CVI 
Criterion  Composites 
and  Combinations 


ASVAB 

CVI  Performance 
ASVAB+CVI  Performance 

Computer  Tests 

ASVAB+Computer  Tests 

ASVAB+Comp.  Tests+CVI  Performance 

AVOICE 

ASVAB+AVOICE 

ASVAB+AVOICE+CVl  Performance 
JOB 

ASVAB+JOB 

ASVAB+ JOB-^-CVI  Performance 

Spatial 

ASVAB+Spatial 

ASVAB+SpatiaUCVI  Performance 
ABLE 

ASVAB+ABLE 

ASVAB+ABLE+CVI  Performance 


CVII  Criterion  Composite 


CTP 

GSP 

ELS 

MPD 

PFB 

.33 

.42 

.11 

-.05 

.11 

.47 

.43 

.30 

.26 

.48 

.51 

.51 

.33 

.26 

.47 

.23 

.13 

-.01 

-.04 

.10 

.37 

.41 

.13 

,05 

.12 

.52 

.51 

.33 

.27 

.46 

.15 

.16 

.06 

-.02 

.06 

.43 

.44 

.14 

.00 

.13 

.54 

.52 

.33 

.27 

.46 

.12 

.00 

.19 

.30 

.12 

.33 

.41 

.16 

.20 

.16 

.51 

.51 

.34 

.31 

.48 

.47 

.41 

.14 

-.01 

.04 

.41 

.43 

.10 

-.06 

.11 

.52 

.51 

.33 

.26 

.46 

.10 

.01 

.21 

.15 

.29 

.34 

.41 

.22 

.12 

.25 

.51 

.52 

.36 

.30 

.47 

Note:  Ns  *  102-121.  Correlations  are  uncorrected  for  range  restriction.  Coefficients  do  rot  require 
shrinkage  adjustments.  CVI  Criterion  Scores  and  Predictor  CcinDosttes  were  sumred. 
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Table  9.9 


Correlations  Between  CVI-Weighted  Predictor  Composites,  CVI  Criterion 
Composites,  and  CVII  Criterion  Composites  for  Residual  Scores:  Computed  on 
Total  Sample _ 


Predictor  and  CVI 

CVII 

Criterion  Composite 

Criterion  Composites 
and  Combinations 

CTP 

GSP 

ELS 

MPD 

PFB 

ASVAB 

.17 

.20 

.12 

,09 

.07 

CVI  Performance 

.35 

.20 

.30 

.21 

.55 

ASVAB+CVI  Performance 

.35 

.21 

.33 

.21 

.54 

Computer  Tests 

.21 

.06 

-.03 

-.04 

.10 

ASVAB+Computer  Tests 

.22 

.16 

.13 

.13 

.07 

ASVAB+Comp.  Tests+CVI  Performance 

.36 

.21 

.33 

.22 

.53 

AVOICE 

.16 

.09 

-.02 

.00 

.03 

ASVAB+AVOICE 

.30 

.23 

.11 

.17 

.09 

ASVA6+AV0ICE+CVI  Performance 

.39 

.23 

.32 

.23 

.52 

JOB 

.08 

-.06 

.16 

.24 

.07 

ASVAB+JOB 

.17 

.19 

.16 

.26 

.10 

ASVAB+J06+CVI  Performance 

.35 

.21 

,34 

.24 

.54 

Spatia 1 

.33 

.19 

.07 

.07 

.04 

ASVAB+Spati a  1 

.23 

.17 

.11 

.06 

.07 

ASVAB+Spatial+CVI  Performance 

.36 

.20 

.33 

.20 

.53 

ABLE 

.03 

-.09 

.18 

.15 

.31 

ASVA8+ABLE 

.17 

.15 

.21 

.22 

.24 

ASVAB+ABLE+CVI  Performance 

.36 

.20 

.36 

.25 

.54 

Note:  Ns  =  102-121.  Correlations  are  uncorrected  for  range  restriction. 
Coefficients  do  not  require  shrinkage  adjustments.  CVI  Criterion 
Scores  and  Predictor  Composites  were  summed. 


value  cf  the  correlation  over  that  obtained  when  the  correlations  are 
calculated  within  MOS  and  then  averaged. 


SUMMARY  AND  CONCLUSION 

In  sum,  these  results  provide  evidence  that  A5VAB  scores,  weighted  on 
the  basis  of  regression  estimates  for  predicting  first-tour  performance, 
predict  second-tour  "can  do"  performance  with  substantive  validity.  The 
results  also  provide  impressive  evidence  of  con-'ergent  and  discriminant 
validity  of  first-tour  job  performance  criteria  for  predicting  second-tour  job 
performance  criteria. 
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Although  the  Trial  Battery  composites,  especially  the  Spatial 
composite,  were  reasonably  successful  at  predicting  second-tour  performance, 
these  test  scores  were  collected  near  the  end  of  the  soldier's  first  tour  and 
not  prior  to  enlistment  as  were  the  ASVAB  scores.  Future  analyses  of  the  LVI 
Experimental  Predictor  Battery  and  I.VII  criterion  scores  will  provide  better 
indications  of  the  new  predictors'  relationships  with  second-tour  performance. 

All  analyses  were  conducted  on  extremely  small  subsamples  from  much 
larger  original  samples.  The  precise  reasons  for  the  sample  shrinkage  are 
largely  unknown  and  caution  must  be  used  in  interpreting  these  results.  On 
the  other  hand,  no  analyses  used  maximization  techniques  that  capitalized  on 
specific  sample  characteristics  and  there  should  be  no  "shrinkage"  in 
correlations  for  those  reasons.  In  addition,  the  pattern  of  relationships 
between  CVII  criterion  construct  scores  and  ASVAB/CVI  Trial  Battery  scores  was 
similar  to  the  pattern  of  relationships  between  the  CVI/criterion  construct 
scores  and  the  ASVAB/CVI  Trial  Battery  scores.  This  similarity  also  argues 
for  the  stability  of  the  findings. 
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Chapter  10 

OVERALL  SUMHARY  AND  FUTURE  PLANS 

John  P.  Campbell 

During  the  second  year  of  the  Career  Force  Project,  the  major  emphases 
were  on  building  the  data  file  for  the  Longitudinal  Validation  cohort, 
developing  the  basic  scores  for  the  final  versioi.j  of  the  Experimental 
Predictor  Battery  and  the  first-tour  performance  measures,  and  carrying  out 
the  basic  longitudinal  validation.  These  activities  are  at  the  core  of  the 
Project  A/Career  Force  design. 

The  Longitudinal  Validation  sample  is  the  major  validation  sample  for 
the  project.  It  permits  a  very  complete  and  stringent  replication  of  the 
estimates  of  predictive  validity  obtained  using  the  First-Tour  Concurrent 
Validation  Sample  (CVI).  It  also  incorporates  the  validation  of  the  ASVAB  and 
the  Experimental  Predictor  Battery  against  the  major  components  of  training 
performance.  Finally,  the  individuals  from  this  cohort  who  reenlisted  have 
been  followed  up  into  their  second  tour,  and  a  major  assessment  of  their 
performance  as  junior  NCOs  is  ongoing.  The  ultimate  payoffs  from  Project  A/ 
Career  Force  will  be  realized  as  a  result  of  analyzing  the  data  file  generated 
by  this  cohort. 


SUMHARY  OF  YEAR  TWO 

For  the  analyses  conducted  during  year  two,  the  major  questions  were: 

(1)  Did  the  Experimental  Predictor  Battery  yield  the  same 
pattern  of  basic  scores  in  LVI  as  it  did  in  CVI? 

(2)  Did  the  measures  of  first-tour  performance  produce  the 
same  model  of  performance  and  the  same  specifications 
for  performance  criterion  scores  as  were  found  in  CVI? 

(3)  In  terms  of  both  the  level  of  predictive  accuracy  and 
the  convergent/discriminant  patter-n  of  validities,  did 
the  results  obtained  in  the  LVI  analysis  replicate 
those  in  CVI? 

While  the  LV/LVy/LVI  cohort  was  the  principal  focus  of  attention,  the 
second-tour  performance  data  collected  at  the  same  time  from  the  Second-Tour 
Concurrent  Validation  sample  (CVII)  permitted  a  preliminary  examination  of  the 
nature  of  performance  in  the  second  tour,  and  the  accuracy  with  which  it  can 
be  predicted  from  the  ASVAB,  from  the  ABLE,  and  from  prior  (i.e.,  first-tour) 
performance.  The  questions  asked  of  the  second-tour  data  were  the  following: 

(1)  Is  the  ASVAB  a  valid  predictor  of  second-tour 
performance?  For  which  performance  components? 

(2)  Does  the  ABLE  add  incremental  validity  to  the 
prediction  of  Sf'cond-tour  performance?  For  which 
performance  components? 
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(3)  Can  second-tour  performance  be  predicted  from  first- 
tour  performance?  Is  there  convergent  and 
discriminant  validity  across  performance  components? 

The  dominant  picture  of  the  results  of  the  second-year  analysis  is  one 
of  almost  unbelievable  consistency  between  the  Concurrent  Validation  and 
Longitudinal  Validation.  The  LV  predictor  scores  exhibited  the  same  factor 
patterns  as  before,  as  did  the  LVI  performance  scores.  For  example,  the 
factor  analysis  of  the  10  Army-wide  performance  rating  scales  yielded  the  same 
three  factors  and  the  factor  loadings  themselves  were  virtually  identical 
between  CVI  and  LVI.  The  model  of  first-tour  performance  that  best  fit  the 
LVI  data  was  the  model  generated  in  CVI.  The  hypothesized  structural  models 
for  training  performance  (LV.)  and  for  second-tour  performance  (LVI I)  that 
were  based  on  the  CVI  model  ntting  analysis  also  received  the  strongest 
support  in  the  confirmatory  tests.  Consequently,  there  is  also  great 
consistency  in  the  way  the  latent  structure  of  performance  is  described  as 
individuals  move  from  the  training  environment  through  their  first  tour  and  on 
into  the  second  tour. 

The  CVI  and  LVI  cohorts  produce  the  same  level  and  pattern  of 
validities,  with  the  partial  exception  of  the  A3Lt‘-ELS  relationship,  which  was 
lower  in  LVI  than  in  CVI.  This  difference  on  the  Eftort/Leadership  element, 
while  not  large  in  comparison  to  the  extant  personnel  selection  literature, 
did  seem  to  result  from  a  somewhat  heavier  weight  given  to  ratings  of 
technical  performance  by  the  LVI  raters  and  a  somewhat  higher  Social 
Desirability  response  set  for  the  ABLE  in  LVI.  The  latter  was  certainly  not 
unexpected,  given  the  mere  "gamelike"  conditions  under  which  ABLE  was 
administe*'ed  in  the  l.V  s<imple. 

One  major  benefit  from  the  extensive  rcanalysis  of  ABLE  scores  by  ARI 
researchers  was  the  set  of  seven  factor  scores  (ABLE  -  114  items)  described  in 
Chaoter  2.  In  both  the  LVI  and  CVII  validations  this  set  of  scores 
outperformed  all  others,  and  used  fewer  items. 

In  general,  the  ASVA8  was  again  shown  to  be  a  very  good  predictor  of  the 
"will  do"  component  of  pei'^'orniance  as  well  as  ?  reasonably  good  predictor  of 
the  extra  effort  and  peer  leadership  and  support  factor  (ELS).  The  ABLE  was 
again  a  good  predictor  of  the  "will  do"  components,  but  was  not  as  accurate 
for  ELS  in  LVI  as  it  was  in  CVI. 

ASVAB  and  ABLE  also  yieldeo  high  validities  for  the  appropriate 
performance  components  in  the  second  tour.  In  fact,  after  the  appropriate 
corrections  1'or  range  restrictions,  the  validities  were  virtually  as  high  in 
CVII  as  they  were  in  CVI  and  LVI,  even  given  the  1  i.mitations  of  the  CVII 
sample.  As  mignt  be  expected,  the  validities  of  the  ASVAB-  and  the 
Exper imenta 1  Battery  for  predicting  training  performance  were  even  higher. 

What  was  even  more  gratifying  was  that  the  differential  pattern  of  va'idities 
across  training  performance  components  was  similar  to  wnat  was  obta^neo  from 
predicting  job  performance  itself. 

finally,  even  though  th..‘  sample  size  was  very  small,  and  be  tween-, "OS 
diffeverces  could  not  be  accounted  for,  the  prediction  of  second-tour 
performance  f^'orn  first-tour  performance  (i.e.,  the  CVI/CVII  samplei  did 
exhibit  substantial  predictive  validity.  It  also  exhibited  the  e/pocteu 
d  i  f  I’er  c*n  t 'a  i  pattern  of  results  across  performance  factors,  i-i  spite  of  tne 
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relatively  high  intercorrelations  among  some  of  the  factors.  This 
predictability  of  performance  from  performance  was  indeed  gratifying  and 
created  a  great  deal  of  interest  in  the  forthcoming  analysis  of  the  data  from 
the  larger  LVIl  sample. 

Again,  the  results  of  the  Project  A/Career  Force  analyses  done  to  date 
exhibit  more  consistency,  in  both  the  level  and  the  pattern  of  predictive 
accuracy,  than  has  ever  been  demonstrated  previously  in  personnel  research. 
Furthermore,  the  tests  of  this  consistency  have  been  more  stringent  than  ever 
before  (e.g.,  the  use  of  the  same  model  of  performance  components  across 
organizational  levels  and  across  samples  from  different  cohorts).  The  basic 
components  of  training  and  job  performance  can  be  predicted  with  considerable 
accuracy,  and  with  very  high  consistency. 


FUTURE  PLANS 

During  year  three,  the  Career  Force  Project  will  concentrate  on  two 
major  activities.  First,  the  Longitudinal  Validation  second-tour  data 
collection  will  be  completed  and  the  data  file  will  be  cleaned  and  edited, 
using  the  procedures  developed  in  and  LVI.  Second,  the  data  analysis  work 
will  go  beyond  the  basic  validation  stage  to  consider  a  number  of  important 
issues  pertaining  to  validity  generalization  and  the  optimal  prediction  of 
performance,  using  (a)  existing  predictors,  (b)  new  predictors,  and  (c)  prior 
performance . 

Consequently,  during  year  three  of  the  Project  considerable  effort  will 
be  devoted  to  developing  the  "optimal'  prediction  equation  for  each 
performance  component  within  each  MOS  under  different  sets  of  constraints, 
such  as  total  testing  time  or  the  extent  to  which  the  content  of  ASVAB  can  be 
changed.  The  next  order  of  business  will  be  to  begin  examining  how  the 
optimal  equation  generalizes  (a)  across  performance  components,  (b)  across 
MOS,  and  (c)  across  organizational  levels.  That  is,  to  what  extent  can 
equations  be  collapsed  across  these  three  parameters  without  losing 
significant  selection  and  classification  information? 

After  the  extent  of  validity  genera  1 izat ion  is  known,  then  we  can  begin 
to  ask  questions  about  how  predictor  information  should  be  weighted,  how 
reenlistment  decision  making  can  be  most  appropriately  earned  out,  what  the 
tradeoff  should  be  between  using  information  for  selection  versus 
classification,  and  what  the  tradeoffs  are  between  maximizing  one  selection/ 
classification  goal  versus  another  (e.g.,  maximize  aggregate  performance  vs. 
minimize  attrition).  These  latter  activities  will  carry  us  well  into  year 
four  and  should  provide  the  Army  with  valuable  information  on  how  to  structure 
Its  personnel  management  procedures  to  best  deal  wit^h  its  radically  different 
environment. 
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LVI  : 


Appendix  A 

TASKS  AND  TESTS  IN  HANDS-ON  AND  JOB  KNOWLEDGE  COMPONENTS 


A-1 


Table 


LVi:  Tasks  and  Tests  In  Hands-On  and  Job  knowledge  Components 


Task* 

HO 

JK 

Functional 

Category 

Task 

Factor 

Task 

Construct 

IIB  Infantryman 

Perform  CPR 

X 

F  irst  Aid 

Safety/Survival 

Technical 

Administer  nerve  agent  antidote-self 

X 

First  Aid 

Safety/Survivdl 

Technica 1 

Apply  field  or  pressure  dressing 

X 

X 

F  irst  Aid 

Safety/Survival 

Technical 

Put  on  Ml  7  mask 

X 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Technical 

Put  on  protective  clothing 

X 

Nuc/Bio/Cheni 

Safety/Survival 

Technica  1 

Identify  terrain  features 

X 

Naviga te 

Basic 

Soldiering 

Technical 

Navigate  on  ground 

X 

Navigate 

Sasic 

Soldiering 

Technica 1 

Estimate  range 

X 

Navigate 

Basic 

Soldiering 

Technical 

Prepare  M60  range  card 

X 

A 

Weapons 

Basic 

Soldiering 

1  c-chnica  1 

Maintain  H16A1 

X 

X 

Weapons 

Basic 

Soldiering 

Technica  ■ 

Load  and  clear  M60 

X 

X 

Weapons 

Basic 

Soldiering 

Technica 1 

Set  headspace/ timing  on  ca'.  .50 

X 

X 

Weapons 

Basic 

Soldiering 

Technica 1 

Zero  AN/PVS-J  to  M16A1 

X 

X 

Weapons 

Basic 

Soldiering 

Technical 

Engage  with  hand  grenades 

X 

X 

Field  Techniques 

Basic 

Soldiering 

Technical 

Call  for  indirect  fire 

X 

F ie Id  Techniques 

Basic 

Soldiering 

Technica 1 

Cot lect, /report  information 

X 

F ield  Techniques 

Basic 

Soldiering 

Technical 

Move  Over  ohstacles 

X 

Field  Techniques 

Basic 

So  Id ier ing 

Technica 1 

Camouflage  self  ana  equipment 

X 

F  ie  Id  Techniques 

Basic 

Soldier  1 ng 

Techn  ica 1 

Hove  under 

X 

F  ie  Id  Techniques 

Basic 

Soldiering 

Technica  1 

Instal  1  c  laymore  .-nine 

X 

X 

F ield  Techniques 

Basic 

Soldiering 

Technical 

Hove  Over  urban  terrain 

X 

X 

F ield  Techniques 

Basic 

Soldiering 

Technica  1 

Select  position  in  urban  terrain 

X 

Field  Techniques 

Basic 

Soldiering 

Technical 

Establish  observation  post 

Field  Techniques 

Basic 

Soldiering 

Technical 

Select  Overwatch  position 

X 

Field  Techniques 

Basic 

Soldiering 

Technica 1 

Operate  AN/PVS-5 

X 

Field  Techniques 

Basic 

Soldiering 

Technical 

Conduct  surveillance 

X 

Field  Techniques 

Basic 

Soldiering 

Technical 

Operate  LAW 

X 

Ant itank/AntiAir 

Basic 

Soldiering 

Technical 

Prepare  Dragon  for  firing 

X 

X 

Antitank/AntiAir 

Basic 

Soldiering 

Technical 

Ooerate  radio  AN/PPC-77 

X 

X 

Cotmwnications 

cunmunicat  ions 

Technical 

identify  armor  voniclcs 

X 

Ident ify 

Ident 

fy  Targets 

Technica  1 

*  Short  task  titles  are  given. 
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Table  A-l 


I.Vl;  Tasks  and  Tests  in  Hands-On  and  Job  Knoxledge  Components  (Continued) 


Task* 

HO 

JV 

functional 

Category 

Task 

Construct 

13B  Cannon  Crevmember 

Administer  nerve  agent  antidote-self 

X 

X 

f  iPSt  Aid 

Safety/Surviva 1 

Basic 

Perform  CPR 

X 

f  irst  Aid 

Safety/Survival 

Basic 

Prevent  shock 

F irst  Aid 

Safety/Survival 

Basic 

Put  on  Ml?  mask 

X 

X 

Nuc/Bic/Chem 

Safety/Surviva 1 

Basic 

Put  on  protective  clothing 

X 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Basic 

Decontaminate  skin  and  equipment 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Determine  azimuth  using  M-2  conpass 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Measure  azimuth  on  map  with  protractor 

X 

X 

Navigate 

Basic  So'diering 

Sasic 

Load  and  clear  M16AI 

X 

X 

Weapons 

Basic  Soldiering 

Sasic 

Set  headsoace/t ining  on  cal  .50 

X 

.X 

Weapons 

Sasic  Soldiering 

Sasic 

Maintain  cal  .50 

X 

X 

Weapons 

Basic  Soldiering 

Sas  ic 

Jse  visual  signals  to  control  movement 

X 

X 

F  ie id  Techniques 

Bas'C  Soldiering 

Sas  IC 

Camouflage  equipment 

X 

f  ield  Techniques 

Sasic  Soldie''ing 

Sasic 

Use  challenge  and  password 

X 

Customs  4  Laws 

Basic  Soldiering 

bas  ic 

Install  and  operate  f-eld  telephone 

X 

X 

Ccmnunicat  ions 

Coirrunicat  ions 

Basic 

Identify  armor  vehicles 

X 

Identify 

Identify  Targets 

'C 

Emplace/ recover  collimator 

X 

X 

Operate  Sights 

Technics  1 

Technical 

Emplace/ recover  aiming  posts 

X 

X 

Operate  Sights 

Technica  1 

Tectinica ! 

Lay  howitzer  for  initial  direction 

X 

X 

Operate  Sights 

Technica  I 

Technica 1 

bores ignt  Ot  telescope  using  DAP 

X 

t 

Operate  Sights 

Technical 

Technical 

Sight  on  target  with  Of  telescope 

X 

X 

Operate  Sights 

Technical 

Technica i 

Oisassemble/assemble  br,.ech  mechanism 

X 

X 

Opera te/Mainta in  Howitzer 

Technica  1 

Technica 1 

Prepare  howitzer  for  operation 

X 

Operate/Maintain  Howitzer 

Technica 1 

Technical 

Prepare  position  for  howitzer 

X 

Operate/Maintain  howitzer 

Technica  I 

Technical 

Prepare  DA  form  2404 

X 

Operate/Ma ihtain  Howitzer 

Technica ; 

Technica 1 

Store  anmunition  for  firing 

X 

Operate/Maintain  Howitzer 

Technica  1 

Technica 1 

Drive  se 1 f -prope 1  lea  howitzer 

X 

Operate/Ma intd in  Howitzer 

Technica  1 

Technica  1 

Drive  prune  mover  for  towed  howitzer 

% 

Operate 'Ma inta in  Hciwitzer 

Technica  1 

’echniral 

Load  anq  fire  preoared  round 

X 

Operate/Ma inta in  Howitzer 

Technica  1 

Technica  1 

Perform  PMCS 

X 

Operate/Maintain  Howitzer 

Technica  1 

Technica 1 

Prepare  anmunition  for  firing 

X 

Operate/Maintain  Howitzer 

Technica  1 

Technica  1 

’  Short  task  titles  a'-e  given. 


A-4 


Tabic  A-1 


LVI:  Tasks  and  Tests  in  Hands-On  and  Job  knowledge  Components  (Continued) 


Ta3i(* 

HO 

Jk 

Eunctional 

Category 

Task 

Factor 

Task 

Construct 

i9E  M50  Amor  Crewman 

App'y  f icld/pre isure  dressing 

X 

X 

P irst  Aid 

Safety/Surviva 1 

Basic 

Adm.nister  nerve  agent  antidote-self 

X 

F irst  Aid 

Safety/Surviva 1 

Basic 

Prevent  shock 

X 

F  irst  Aid 

Safety/Surviva 1 

Sasic 

Put  on  H25AI  mask 

X 

Huc/Bto/Cliem 

Safety/Surviva ) 

Basic 

Put  on  protective  clothing 

X 

Nuc/Blo  Chem 

Safety/Surviva 1 

Basic 

Prepare  vehicle  for  nuclear  attack 

X 

Nuc/Bio/Chcm 

Safcty/Surv i va 1 

Basic 

Identify  terrain  features 

X 

Nav  igate 

Basic  Soldiering 

Basic 

Determine  grid  coordinates 

X 

X 

tiav  iqate 

Basic  Soldiering 

Basic 

Maintaifi  HjAl 

X 

X 

veapons 

Basic  Soldiering 

Bas  1C 

Ma 1 nta in  M24h  cnax 

X 

X 

we-iocns 

Basic  Sold'enng 

Bas  ic 

Ha inta in  .4S  ca 1  pisto 1 

X 

Weapons 

Sasic  So'dienng 

Bas  ic 

Collect/report  information 

X 

^  le  id  Techniques 

Basic  Soldiering 

Basic 

I  ,ns:a  1 1  c  layrore  mine 

X 

Field  'echniques 

Basic  Soldiering 

Basic 

Identify  minefield  markers 

X 

Fieiq  Techniques 

Basic  Soldiering 

Basic 

Know  r-ghts  as  POW 

X 

LustoiTis  S  Laws 

Basic  Soldiering 

Basic 

use  auto'Jted  CnOI 

K 

X 

Corrmnicat  ions 

Conmjnicat  ions 

Basic 

Send  radio  message 

X 

X 

fcfnnjni rat  ions 

Cormunicat  ions 

Pas  ic 

Operate  radio  ah;vRC-64 

1 

J 

Ccmnunicjt  ions 

Coiminicol  ions 

Bas  IC 

Identify  armor  vehicles 

X 

Identify 

Idem ! fy  Targets 

Basic 

Operate  gas  particulate  filter  unit 

V 

X 

Operate  Tanks 

Technical 

Technical 

Start/stop  tank  engine 

X 

X 

Operate  Tanks 

Technica  1 

Technical 

Remove/ ins ta I i  track  tio'.k 

X 

X 

Operate  Tanks 

Technical 

Technical 

Escape  from  tank 

X 

X 

Operate  Tanks 

Technical 

Technical 

Extinguish  fire  In  tank 

X 

Operate  Tanks 

Technica 1 

Technical 

Bores igtit/system  calibrate  H00A3  tank 

X 

X 

Tank  Gunnery 

Tcchnlca  1 

Technical 

Prepare  loader's  station  for  operation 

X 

X 

Tank  Gunnery 

Technica  1 

Technical 

Perform  prepare-  to- f ire  checks 

X 

X 

Tank  Gunnery 

Technica  1 

Technica 1 

Load 'on load  mam  gun 

X 

Tank  Gunncrv 

teciinica  1 

Technica  1 

Engage  targets  with  irjin  gun 

X 

Tank  Gunnery 

Technica 1 

Technical 

Apply  ansfire  procedures 

X 

Tank  Gunnery 

Technica 1 

Technica 1 

*  Short  task  titles  are  given. 
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LVI;  Taski  and  Tests  in  Hands-On  and  Job  Knowledge  Components  (Continued) 


Task* 

HO 

JK 

functional 

Category 

Task 

factor 

Task 

Construct 

19K  Tank  Crewnvan 

Apply  field/pressure  dressing 

X 

X 

first  Aid 

Safety/Surviva 1 

Basic 

Administer  nerve  agent  antidote-self 

7. 

first  Aid 

Safety/Surviva 1 

Basic 

Put  on  protective  clothing 

X 

X 

Nuc/8io  Cheir. 

Safety/Survival 

Basic 

Conduct  hasty  decontamination 

X 

Nuc,'Bio/Chem 

Safety/Surviva 1 

Basic 

Conduct  umrasking  procedures 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Bas  ic 

Use  H8  chemical  alarm  system 

X 

Nuc/8io/Chem 

Safety/Surviva 1 

Basic 

Determine  grid  coordinates 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Determine  location  by  association 

X 

Navigate 

Basic  Soldiering 

Bas  ic 

Analyze  terrain  using  five  aspects 

X 

Navigate 

Basic  Soldiering 

Bas  1C 

Maintain  cal  .50  H2  H8  machinegun 

X 

X 

Weapons 

Basic  Soldiering 

Bas  ic 

Ma inta in  M240  coax 

X 

X 

Weapons 

Basic  Soldiering 

Dds  1C 

Maintain  .45  cal  pistol 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Identify  minefield  markers 

X 

f  ie  Id  Techniques 

Basic  Soldiering 

Bas  1C 

Use  challenge  and  p.’.ssword 

X 

Customs  &  Laws 

Basic  Soldiering 

Basic 

Use  automated  CEOI 

X 

X 

Comninicat  ions 

Conninications 

Bas  L 

Send  radio  message 

X 

X 

Conwjnicat  'ons 

Cc.mikjh' cat  ions 

Bas  ic 

Estab i  isn/enter,' leave  radio  net 

X 

X 

Comnkjnicat  ions 

Cormun  leaf  ions 

Bas  ic 

Identify  ar.mor  vehicles 

X 

Ident i fy 

Identify  Targets 

Bas  ic 

Trouble. hoot  tank  systems 

X 

operate  Tank 

Techmea  i 

Terhn ica 1 

Recover  an  Ml  tank  by  similar  vehicle 

X 

Operate  Tank 

Technica 1 

Technical 

Manually  remove  mam  gun  round 

X 

Operate  Tank 

Technica 1 

Technica 1 

Bores ight/system  calibrate  Ml  tank 

X 

X 

Operate  Tank 

Technica 1 

Tecnnical 

Prepare  driver's  station  for  operation 

X 

X 

Operate  Tank 

Technical 

Technica 1 

Perform  loader's  prep-to-fire  checks 

X 

Operate  Tank 

Technica 1 

Technical 

Loaa/unload  105-mni  main  gun 

X 

Operate  Tank 

Technica 1 

Technical 

Operate  airnio  doors  manually 

X 

X 

Tank  Gunnery 

Technica 1 

Technical 

Perform  before  operations  PMCS 

X 

X 

Tank  Gunnery 

Technica 1 

Technical 

Operate  AN/VVS-2  in  loader's  hatch 

X 

X 

Tank  Gunnery 

Tecnnica 1 

Technical 

Perform  operator  maintenance  on  M9 

X 

X 

Tank  Gunnery 

Technica 1 

Technical 

Use  visual  sicmals 

X 

X 

Tank  Gunnery 

Technica 1 

Technical 

Evacuate  wounded  crewman  from  Ml  tank 

X 

lank  Gunnery 

Technica 1 

technical 

*  Snort  task  titles  are  given. 


I 


A- 6 


Tsble  A-i 


I.VI:  Tasks  and  Tests  in  Hands-On  and  Job  Knowledge  Ccn'icnenti  (Continued) 


Fi'.ictiorat 

T  jS> 

Task* 

HO  JK 

C,Htegorv 

Fartoi 

LGr.5  tr'^i. ' 

Apply  f  ield/prcssere  dressing 

X 

X 

f  'rst  Alt! 

Safcty/S'irv  iva  1 

lias  ic 

Parfonr.  CPR 

X 

First  Aid 

Siifety/'.iurv  iv3 1 

Has  ic 

Put  on  protective  clothing 

X 

X 

Nuc/Hic/Cnem 

Safety/Surv  iva! 

Eas'c 

Cecontaninate  skin  and  equipment 

X 

NiiC.'Cio  -  Chem. 

baf  cty/Surv iva  1 

Sas  IC 

Hainta 'n  Ml  7  mask 

X 

Niic/3io,'Cnem 

Sa  ’ety.'Surv  i  va ' 

Basic 

Determine  grid  coordinates 

X 

X 

Navigate 

Basic  .Eoldierinq 

Bds  ic 

Load  ana  clear  M'.6A1 

X 

X 

Weapons 

Basic  Sc  Idler  Tig 

ta  s '  c 

Hainta in  Ml6Ai 

X 

weapons 

Basic  Soldiering 

nas  ■  c 

P’-jctice  no  1  se/ 1  ight  / 1  i  f  ter  discipline 

X 

‘^ieiq  Tccnnioues 

Basic  Soldiering 

Ei  j  s  ‘  c 

Know  >our  rights  as  POH 

X 

Custor-a  A  td-vs 

fasic  Soldifcrii.g 

Das  1  c 

tstabl  isn.enter/ leave  radio  net 

X 

CcTunirat  .ons 

Cor.rumcat  ions 

Basic 

Use  K^C  idCO  C  iper/Aethent  icat  ion  5>ste(r 

< 

X 

Cormiricnt  ip-.v 

ilqrnun icat  ions 

(iasic 

Identify  armar  vehicirs 

X 

ident  ify 

Identify  Targets 

Basic 

Pe^^orm  PHCf-  on  t'ccs 

% 

X 

Dr  1  ve 

Ve^ir  ies 

Basic 

Pecoanite  jra  ito lenient  tCCH 

< 

Teletype  Opera t litr^ 

technica  1 

Technica  i 

prepare  ressace  in  16- 1  me  forirvu 

X 

X 

Tc  letKpe  Operaticns 

Tectinica  1 

Tpc‘';iica  I 

Ocerate  in  radio  nets 

X 

Teletype  Operaticns 

Techmea  1 

Technica  I 

Handle  classified  equipment  i  mat  trial 

X 

Teletype  Operations 

'rficnnica  1 

Tecnnic.i  1 

install  rajiote letype 

X 

X 

Install  Teletype  Cquipmer.t 

Techn’ca  1 

Technica  ' 

irsta  1 1  radio  set 

X 

X 

Install  Teletype  Equipment 

Technica I 

Ttcnnica  i 

Erect  doublet  antenna 

X 

Install  Teletype  Equipment 

Terhn  i  ta  1 

Teciimca  1 

Erect  expedient  antennas 

X 

Install  Teletype  Equipirent 

Tecmica  1 

Techn  ice  1 

•iperate  rad lote letyoe 

1 

X 

Operate  Teletype  Equipment 

■'i.'cnnira  1 

technica  1 

jcerato  terminal  cnmio  set 

X 

X 

Ooerate  Teletype  Equipment 

Tec'imca  i 

Toch’iica  1 

ocerate  radio  control  group 

X 

Operate  Teletype  Equipment 

Trr-.tiu'.a  : 

Tecnmca  1 

TrouD lesnoot  radiote letipe 

X 

Maintain  Teletype  Equipment 

'senr  ica  I 

Tncnn  .ca  i 

Trruplesnoot  radio  set 

X 

Maintain  leletype  fg>:  pment 

Tacnnica  1 

1 eenn ica  ’ 

Perform  CflCS  on  terminal  ucirm  set 

X 

Maintain  leietvpe  Egupment 

I'echn  ica  1 

Teennir,  ; 

Ocerate  generator  set  ?!i-620 

X 

1 

Oenerators 

Techniral 

Techn'ca  > 

I'rdjS  Icsr.cc*  genera tO'  set  PU-6aC 

t 

X 

Geiidi  diors 

’■prt'n  1 1','  1 

’  lr;5  i 

*  Snort  tasK  title’  are  giuen. 


Table  A-; 


LV'I:  Tasss  and  Tc5  .n  -0'^  and  j  ;o  K.i-av ;  ci-.^e  Co'.i.:)n‘-*nii  (Cunt  .nued) 


r  j.irt  lone  1 

Task 

Task 

r't)  JK  ilateqc.'y 

Factor 

Construct 

^•r  ..-.'jn!  iheei  vemclc  Msehanic 


unpiy  ♦  le id/pressure  dressing 

X 

X 

F irst  Aid 

Sa  fe  ty/Surv i va  1 

Bas  ic 

Administer  ner'..n  agent  ant  ioote-caiua  i  ty 

% 

First  AtO 

Sofety/Surv iva 1 

Bas  IC 

Fr*  on  protcct’ve  clctnirg 

X 

Su  :/8''j/ChfT 

Safety/Surv iva 1 

Bas  ic 

Tut  on  M]  ?  TV"  5k 

X 

X 

nuc/BiO/CneP 

Safr.ty/Siirv!v3  1 

Bas  tc 

Dctennine  ar;(j  vy;’ ’  "ates 

X 

Sj.1  lOkte 

Basic  Soldering 

Bas  IC 

tV'ermine  svagnctic  atinutt  «./'corcass 

« 

X 

ha-vigcte 

Basic  Soldiering 

Bas  It 

Ma 'nta in  M16A1 

X 

Weapons 

Basic  Soldiering 

Bas  IC 

Load  and  cisar  mica; 

X 

X 

Weapons 

Basic  Soldiering 

Bas  IC 

C.’-C'- '  iiije  eguicr^nt 

X 

F  ield  Techniques 

Basic  Soldiering 

Basu; 

Lv;  cOaiier.gr  e.ro  passnord 

X 

X 

Cuytomi  i  Lawb 

Casic  Soldiering 

Bas  1C 

M^r-tai r.  assioned  toolkit 

X 

X 

'vemole  l-ecoverv 

venic  le 

Bas  ’C 

f  '-fonn  expedient  repairs 

X 

lei; ’c:e  recovery 

Vehicle 

Sasic 

s'a-.e  star;  O'.ejslrd  voO’cle 

X 

Vehicle  Recoverv 

vehicle 

Bas  1C 

Tow  disanied  ve'ii'le  w/s  tun  wrec-er 

y 

Vehicle  Recovery 

Vehic  Ic 

Basic 

PerforP  annna '  PdC5  (truck) 

X 

Vehicle  Recovery 

Vehic  le 

Basic 

r.eiMM-  e •  w-r-'-'g 

X 

X 

electrical  System. 

fechnica' 

Technical 

'rcLO iosn.tc '  eiectricai  s/ste" 

X 

X 

£  iectntai  System 

Technic.al 

Technica  i 

K»;DisJk:e  srar  lei' 

% 

[lectricai  System. 

Technica  I 

Technica  I 

Pes  ure  ai’  uydraoi  r  oylir.id’' 

X 

X 

Brake. ''Steer  ’  nq.'Suspens  ion 

Technica  I 

Tcchn  ica  1 

Fryiace  w.eel  tearings 

X 

X 

B'-ane.'Steering/Suspen'on 

Technica  1 

Technica  I 

^nv-'-'ice  scr-.  :c«'  brakes 

X 

X 

6ra>.e,' Steering,  Scsoens  icn 

Terhmca  i 

Technica  1 

■•■o.i.o  iesneot  service  Drake  ma  itonct  tens 

X 

X 

Brake/Steering/Suspens  ton 

Technica  1 

Technica 1 

Ti’dDSiesr.y!;:  cr.ike  system  .nkdlfunct’ons 

X 

Brake.'Steering/Suspension 

Technica  1 

Technical 

tro'jcle.-ncut  steering  system 

X 

Brake/Steer i ng/Susoens ion 

Technica! 

Teen,  cal 

Prtlacc  fuel  punp 

X 

X 

Fuel.'Cooi  ing/Liiprication 

Technica  1 

'ecltnica  1 

^-■O'jD lesDoc;  fec'  System  TV  1  fo.-ic;  ions 

X 

X 

Fve  1,  Coo ling/lubr icat lo.n 

Technica  1 

Technica  1 

1  '.esnocit  eng-ne  cooling  system 

X 

F  je  1.  Ccol  ing.'Lubr  icat  ion 

Tccnr.ica  ' 

Technica  1 

Feoiace  raciatcr 

X 

Fue 1 -Cco i ing/L jor  >ra t ion 

T‘'-:nnic2  > 

Technica ' 

Triijrr les-'^oot  engine 

X 

Power  Train/C’iitch 

“ccbnica • 

Tec-mn  ica  1 

Inert  talk  t:t‘es  Jre  aiVL>;:. 


M-0 


Table  A-1 


LVI:  Tasks  and  Tests  tn  Hands-On  and  Job  kno»iedge  Components  (Continued) 


Task* 

HC 

JK 

Functional 

Category 

Task 

factor 

Task 

Construct 

71L  Administrative  Soetiaiist 

Apply  f ield/pressure  dressing 

X 

f  irst  Aid 

Safety/Surviva 1 

Basic 

Administer  nerve  agent  antidote-self 

\ 

First  Aid 

Safety/Surv iva ' 

Basic 

Put  on  protective  clotbir.g 

X 

Nuc/Bio/Cbem 

Safety/Surviva! 

Bas  ic 

Put  on  M17  mask 

X 

X 

Nuc  oio/Chem 

Sd fety/ Surv ' va  * 

Bas  ic 

Ma inta in  M! ?  mask 

X 

Nuc,'6io/Cbem 

Safety/Surv iva 1 

Bas  !C 

Dete'^mine  grid  coordinates 

X 

X 

Nav  igate 

BaS’C  Soloierirg 

Bas  1C 

Determine  magnetic  az  inxjth  «/ compass 

X 

X 

Nav  igate 

Basic  Soldiering 

Bas  1C 

Maintain  H16A1 

X 

X 

Weapons 

Basic  Soldiering 

h.is  >c 

Load  and  c  lean  M16A1 

X 

Weapons 

Basic  Soldiering 

Ba  i  5  s. 

Ca.'iTouf  lane  self  and  eguiorvnt 

X 

cield  Techniques 

Basic  Soidierinq 

5ds  ;c 

Practice  no ise,' 1  ignt/ 1  1 1 ter  discipline 

X 

Field  Techniques 

Basic  Soldiering 

bas  1C 

Know  -ignts  as  PO" 

X 

Customs  4  Laws 

Basic  So  idle- mg 

Basic 

Request  publ  ica  t  icns/b  lank  forms 

X 

Forms,- f  1  les  management 

Techr i ca 1 

lechnica  1 

f  '  le  documents  and  correspondence 

X 

X 

Forms/'files  nanagement 

Technica  1 

Tecnn  ica  i 

Type  straight  copy  material 

X 

Correspondence 

Correspondence 

Technica  1 

Type  nilitary  orders 

X 

X 

Correspondence 

Correspondence 

Techr ica  1 

Type  second  corrKnf  to  Of 

X 

X 

Ocrresoonoence 

Corresponoen-.e 

Tecnn  ica  1 

Tyre  a  joint  messaqeform 

X 

X 

Correspondence 

Correspondence 

Techr  ica  I 

Type  memoranauri 

X 

z 

Correspondence 

Cor'esponce.nce 

Tecnn  ica  1 

Type  oasic  cornnent  to  QC 

X 

X 

Correspondence 

Correspondence 

Tecnn  ica  i 

Dispatch  Outgoing  distribution 

X 

Correspondence 

Correspondence 

Technica  i 

Assemble  correspondence 

X 

Correspondence 

Corr?'spc"'wC'"5*c 

,  -t,  «  ,  u.  ,  1 

•  . wo  I 

Control  Supplies 

X 

Supervision/coord i nat ion 

Technica 1 

Technica 1 

Receive/contro!  office  equipment 

X 

Supervi sion/coord  mat  ion 

Technica  1 

Technical 

Rece'ot/ t'^ansfer  ciassificq  nateriai 

X 

X 

Classified  material 

Technica 1 

Techn 1  cal 

Sateouard  fClO  maten'al 

X 

C  lass  if ied  mater ’a  1 

Technica  1 

Techn 1 ca  i 

*  Sho'-t  task  titles  are  ^iven. 
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Table  A-1 


LVI:  Tasks  and  Tests  in  Hands-On  and  Job  Knowledge  Components  (Continued) 


Task* 

HO 

JK 

Fuortionjl 

Category 

Task 

Factor 

Task 

Construct 

88M  Motor  Transport  Operator 

Apply  field/pressure  .  eSSing 

X 

X 

f irst  Aid 

Safety/Survival 

Basic 

Administer  nerve  agent  antidote-self 

X 

X 

first  Aid 

Safety/Survi.al 

Basic 

Administe-  nerve  agent  antidote-casualty 

X 

X 

First  Aid 

Safety/Survival 

Basic 

Perform  CPR 

X 

First  Aid 

Safety/Surv iva 1 

basic 

Put  on  protective  clothing 

X 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Basic 

Put  on  M17  mask 

X 

X 

Huc/Blo/Chem 

Safety/Surv iva 1 

Basic 

Decontaminate  skin  and  equipment 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Basic 

Oeccntaminate  equipment  w/ABC  MU 

X 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Basic 

Use  MB  detector  paper 

X 

X 

Nuc/Bio/Chem 

Safety/Surviva 1 

Basic 

Determine  grid  coordinates 

X 

X 

Navigate 

Basic  Soldiering 

8as  1C 

Measure  distance  on  a  trap 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Maintain  H16A1 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Load  and  c lear  M16A1 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Load  and  dear  M60 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Camouflage  equ.pment 

X 

field  Techniques 

Basic  Soldiering 

Basic 

Col  lect/reporf  inforrrdtion 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Use  challenge  and  password 

X 

Customs  &  Laws 

Basic  Soldiering 

Basic 

Visually  Identify  thre.  f  aircraft 

X 

Ident  ify 

Identify  Targets 

Basic 

Couole  semi tra i ler 

X 

Drive 

Vehicles 

Basic 

Uncouple  semitrai ler 

X 

Orive 

Vehicles 

Basic 

Operate  tractor  and  semitrailer 

X 

X 

Drive 

Vehicles 

Basic 

Perform  operatcr/crcw  ?KCS 

X 

Orive 

vehicles 

Basic 

Ocerrte  vehicle  in  convoy 

X 

Orive 

Vehicles 

Basic 

Operate  vehicle  off  road 

X 

Or  1  ve 

Vehicles 

Basic 

Operate  vehicle  in  snow  and  ice 

X 

Orive 

vehicles 

Basic 

Transport  general  cargo 

X 

Orive 

Vehicles 

Basic 

Drive  vehicle  under  blackout 

X 

Orive 

Vehicles 

Basic 

Use  defense  whe"  amPuShed  or  attacked 

X 

Drive 

Vehicles 

Bask 

Till  out  Sf  91  (Accident  Repcr') 

.X 

Drive 

Vehic les 

Basic 

•■errorn  vehicle  emergency/ recovery 

X 

Drive 

Vehic  les 

Bas  1C 

“  Short  task  titles  are  given. 
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Uble  A-l 


LVl:  Tasks  and  Tests  in  Hands-On  and  Job  Knowledge  Components  (Continued) 


Task* 

HO 

JK 

Functional 

Category 

Task 

Construct 

91A  Medical  Soecialist 

Apply  Field/pressure  dressing 

X 

X 

First  Aid 

Safety/Survival 

Basic 

Perform  CPR 

X 

X 

First  Aid 

Safety/Survival 

Basic 

Splint  a  suspected  fracture 

X 

X 

First  Aid 

Safety/Survival 

Basic 

Prevent  shock 

X 

First  Aid 

Safety/Survival 

Basic 

Evaluate  a  casualty 

X 

First  Aid 

Safety/Survival 

Basic 

Replace  filters  in  M17  protective  mask 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Put  on  protective  clothing 

X 

Muc/Bio/Chem 

Safety/Survival 

Basic 

Decontaminate  skin  and  equipment 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Determine  grid  coordinates 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Load  and  clear  M16A1 

X 

Weapons 

Basic  Soldiering 

Basic 

Move  over  obstacles 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Practice  noise/light/litter  discipline 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Identify  armor  vehicles 

X 

Identify 

Identify  Targets 

Basic 

Conduct  PMCS 

X 

Drive 

Vehicles 

Basic 

Open  the  airway 

X 

X 

Clinic/Vard  Treatment 

Technical 

Technical 

Initiate  field  medical  card 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Treat  for  shock  (hypovolemic) 

X 

Clinic/Hard  Treatment 

Technical 

Technical 

Manage  patient  with  IV 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Assist  with  sick  call  procedures 

X 

Clinic/Hard  Treatment 

Technical 

Technical 

Initiate  an  IV  infusion 

X 

X 

Clinic/Hard  Treatment 

Technical 

Technical 

Measure  and  record  pulse 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Measure  and  record  respirations 

X 

X 

Clinic/Hard  Treatment 

Technical 

Technical 

Assemble  needle/syringe 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Measure  and  record  blood  pressure 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Change  sterile  dressing 

X 

X 

Clinic/Hard  Treatment 

Technical 

Technical 

Administer  an  injection 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Establish/maintain  a  sterile  field 

X 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Triage 

X 

Clinic/Ward  Treatment 

Technical 

Technical 

Draft/file  TPR  charts  (SF  511) 

X 

Clinic/Ward  Management 

Technical 

Technica  1 

*  Short  task  titles  are  given. 
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Table  A-l 


LVI:  Tasks  and  Tests  in  Hands-On  and  Job  Knowledge  Components  (Continued) 


Task* 

HO 

JK 

Functional 

Category 

Task 

Construct 

958  Military  Police 

Apply  field/pressure  dressing 

X 

X 

First  Aid 

Safety/Survival 

Basic 

Perform  CPR 

X 

First  Aid 

Safety/Survival 

Basic 

Decontaminate  skin  and  equipment 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Put  on  HI'7  mask 

X 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Maintain  M17  mask 

X 

Nuc/Bio/Chem 

Safety/Survival 

Basic 

Determine  magnetic  azimuth  w/compass 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Estimate  range 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Navigate  on  ground 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Determine  grid  coordinates 

X 

X 

Navigate 

Basic  Soldiering 

Basic 

Load  and  clear  M16A1 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Load  and  clear  M60 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Engage  targets  with  M16A1 

X 

Weapons 

Basic  Soldiering 

Basic 

Maintain  M9/cal  .45/cal  .38 

X 

X 

Weapons 

Basic  Soldiering 

Basic 

Call  for/adjust  indirect  fire  » 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Camouflage  self  and  equipment 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Move  under  direct  fire 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

React  to  hostile  fire  during  convoy 

X 

Field  Techniques 

Basic  Soldiering 

Basic 

Prepare/operate  FH  radio  sets 

X 

Corrmunications 

Communications 

Basic 

Use  automated  CEOI 

X 

Conmunications 

Conmunications 

Basic 

Identify  armor  vehicles 

X 

Identify 

Identify  Targets 

Basic 

Perform  operator^crew.PMCS  ‘ 

X 

X 

Drive 

Vehicles 

Basic 

Respond  to  domestic  disturbance 

X 

Respond  to  Alarms 

Technical 

Technical 

Respond  to  traffic  accident 

X 

Respond  to  Alarms 

Technical 

Technical 

React  to  sniper  fire 

X 

Respond  to  Alarms 

Technical 

Technical 

Decide  when  to  use  force 

X 

Conduct  MP  Procedures 

Technical 

Technical 

Requirements  for  lawful  apprehension 

X 

Conduct  MP  Procedures 

Technical 

Technical 

Use  hand/arm  signals  tci  direct  traffic 

X 

X 

Patrol  Duties 

Technical 

Technical 

Preoare  MP  reports  and  forms 

X 

X 

Patrol  Duties 

Technical 

Technical 

Operate  dismount  point  ^ 

X 

X 

Patrol  Duties 

Technical 

Technical 

Collect  and  process .evidence 

X 

Patrol  Duties 

Technical 

Technical 

Perform  patrol  duties 

X 

Patrol  Duties 

Technical 

Technical 

*  Short  task  titles  are  given. 


4 


4 
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LVl  HANDS-ON  TESTS:  AMOUNT  OF  MISSING  DATA 


IVI  Hands  On  -  I  IB  Aanunt  of  Hissing  Data  by  Task  and  Step  (percent) 
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XMAPi  -  Put  on  field  Otesstng  XMBS  =  Load/Reduce/Clear  M60  XH86  •  Cai  .SO  Headspace./ 1  imlng  H189  •  Engage  Targets  with  Grenades 

FHbA  -  Prepare  Uragoii  for  firing  fllBB  *  M60  Range  Card  FHBC  -  Engage  Targets  with  I  AH  XHOl  -  Put  on  H17  Mask 

ft:[4  -  Operate  Radio  AN/PRC-77  XHIl  •  Insta I t/f  ire  MlB  Claymore  FHJ!  -  Urban  terrain  Movement  XIIBA  •  M16  Operator  Maintenance 

FHll  ■  Zero  Ali/PVS-<  on  MlB 


Ijble  B? 

I  VI  lijnds  On  -  13B  ([rack  ni09)  Awunt  of  Hissing  Data  by  Task  and  St^  (percent) 
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OHI?  •  Use  Visual  Signals  OHM  -  Emplace  Aiming  Posts  CHI3  •  Collimator  (H109.  MllO)  0HI5  -  Boresight  Tel.-OAP  (M109) 

01116  -  Sigh;  on  Target  (M109)  OKI?  -  Lay  Hcwitier  (h\09.  WHO)  OHJl  -  Breech  Hech.  (H109)  XHD3  -  Put  on/Hear  HOPP 


LV!  Hands  On  -  138  (Track  KMO)  teount  of  liata  Kissing  by  Task  and  Step  fpercenf ) 
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tVl  >Unds  On  -  13U  ((rack  H198)  taount  of  liala  Hissing  by  Task  and  Step  (percent) 
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one -I  •  Neasur  .'  A/inu'h  with  Ccsipass  0IIC5  ■  Measure  Azimuth  with  Protractor  *H01  •  Put  oi  N17  Mask  0H£3  -  Install  Field  Telephone 

GH12  -  Use  Vi  ual  Signals  OHM  -  Emplace  Aiming  Posts  CHIJ  •  Colllrator  (H198)  OHIL  •  lay  Howitzer  (H198) 

OHIM  •  floresi  iht  lel.-OAP  (M198)  OHIN  ■  Sight  on  Target  (M198)  0HJ6  •  Breech  Mech  (M198)  XHD3  •  Put  on/Wear  MOPP 


l¥l  Kands  fti  -  I9t  Aacunt  of  Oat*  Hissing  by  Tasi  and  Step  (percent) 


B-7 


IVt  lUods  On  -  ]9K  Aaount  of  Utta  Hissing  by  last  and  Step  {pcrfent) 
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JHK6  -  Operate  Aram  Ooors  Hanual'.y  XHKS  -  H2A0  Operators  Haintenance  O'**  -  Visual  Signals  JHHS 

JHBD  *  Cal  .50  Ooerator  Maintenance  JHK8  -  6oresight  HI  lank  J.'WS  -  Prepare  Driver’s  Station 


I VI  lldnds  On  -  31C  taunt  of  Ddta  Hissing  by  lask  and  Step  (percent) 
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B-ll 


LVl  iUnds  On  -  7IL  taount  of  0<ita  Missing  l>y  Task  and  Step  (percent) 


'«»'CO^CL*— «  OJ  r««  ^  sn  lo  OO  Tv  O  —  rsa  ^ 

— I— (%j  r>j  Oij  c-j  Cj  t-si  fsj  fs*  f'l  ro  v  v  *T  ^ 


XHA4  -  Put  on  1  leld  Dressing  XHB^  -  M16  Operator  Maintenance  XHC?  -  Grid  Coordinates  XHOl  •  Put  on/Kear  BU  Mask 

AHHl  •  File  Documents  AtlH3  -  Request  forms-DA  4569  AH.)?  -  Type  Military  Orders  AIIJ3  -  Type  Second  Conment  to  OF 

A.'Ua  ’  Type  Joint  Mcssagefomi  Al)J6  -  Type  Hemrandum  AHJ?  -  Type  Basic  Corment  to  OF  AHK2  -  Receipt  Classified  Mat. 


LV1  Hands  On  -  88H  ^.aQunt  of  a  Missing  by  Task  and  S(cf>  (pLicent) 


XHAl  •  Buddy-Aid  XI1A3  -  Nerve  Agent  Antidote  XHAA  ■  Put  field  Dressing  XHB3  -  Load/Reduce/Clear  M16 

Xie4  ■  H!6  operator  Maintenance  XH85  -  load/Reduce/C tear  M60  XttC?  •  Grid  Coordinates  XHOl  •  Put  on/Wear  M17  Mask 

XlC3  -  Put  on/Wear  MOPP  CHC3  -  Measure  Distance  on  Hap  C1I05  -  Cecontarainate  Equipment  CH06  •  10  Chemical  Agent 

CHH3  .  Couple  Semitrailer  CHHa  »  Uncouple  Semitrailer  CHHS  -  Operate  fractor  &  frailer 


LVl  Hands  On  -  9iA  Xaount  of  Data  Hissing  by  Task  and  Step  (percent) 
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Appendix  C 

EXAMPLE  OF  THE  REPORT  ON  IMPUTATION  OF 
STEP-LEVEL  DATA 
(HANDS-ON  TASK  IN  WOS  71 L) 


C-1 


PROMAH  AUTHORS)  lAURESS  L.  HISE 

DONALD  H.  HCLAUOKLXN 

AMERICAN  INSTITUTES  FOR  RESEARCH 

P.O.  BOX  1113 

Palo  Alto,  CA  94302 


VERSION'  10 
RELEASED'  APRIL  19<9 


CONTENTS 


REPORT  t1 s  MISSINO  DATA  FREQUENCIES  AND  UNIVARIATE  STATISTICS 

REPORT  12'  CHARACTERISTICS  OF  CASES  HITH  HISSINO  VALUES 

REPORT  «3>  CORRELATIONS  BETHEEH  REPORTED  VALUES 

REPORT  84  I  REGRESSION  EQUATIONS 

REPORT  IS'  CONDITIONAL  DISTRIBUTIONS 

REPORT  Ui  (NOT  FULLY  IMPLEMENTED)  ERROR  ANALYSIS 
Input  FU«>  OHJ.DlSt.ITUXHOI 
Output  Flip'  RK0.1MP.I71LXH01 
TITLE  VERSION  IMPUTE. TEMP  ON  PROC. IMPUTE  I71LXHD1 
MISSINO  DATA  REPORT  11'  HlSSlNO  DATA  FREQUENCIES  AND  UNIVARIATE  STATISTICS 


VARIABLE 

N  MISS 

X  MISS 

N  PRES 

MIN  MAX  1/D 

NV 

MEAN 

STD 

M7XHT0TT 

It 

2.48 

634 

13.1730  93. 

0740  D 

6 

57.9161 

12.3483 

STEP01 

181 

26.93 

491 

0. 

1.  1 

2 

0.989817 

0.100397 

STEP02 

1S1 

24.93 

491 

0. 

1.  1 

2 

0.B14664 

0.388570 

$TEP03 

181 

26.93 

491 

0. 

1.  I 

2 

0.802444 

0.398195 

STEP04 

181 

26.93 

491 

0. 

1,  I 

2 

0.755601 

0.429730 

STEP05 

184 

27.38 

488 

0. 

1.  I 

2 

0.399328 

0.496929 

STEP06 

181 

26.93 

491 

0. 

1.  I 

2 

0.914460 

0.279683 

STEP07 

18« 

27  .48 

486 

0. 

1  .  I 

2 

0.753086 

0.431216 

STEPOB 

191 

28.42 

481 

0. 

1.  1 

2 

0.399010 

0.488853 

TITLE  VERSION  IMPUTE, TEMP  OH  PROC. IMPUTE  I71LXMD1 
MISSINO  DATA  REPORT  BZ <  CHARACTERISTICS  OF  CASES  MITH  MISSINO  VALUES  (FOR  EACH  VARIABLE) 

MISSINO  SECOND  VARIABLE 


VARIABLE 

M7XMTOTT 

STEP01 

STEP02 

3TEP05 

STEP 04 

STEP09 

5TEP06 

STEP07 

STEP08 

M7XHTOTT 

57.9161 

1  . 

O.SOOOOO 

0.900000 

0.900000 

0. 

1 

0.500000 

12.3483 

0.  0 

.500000  0 

.900000  0 

.900000 

0. 

0. 

0.300000 

0. 

494 

2 

2 

2 

2 

2 

2 

2 

1 

0.000 

0.232 

0.232 

0.232 

0.232 

0.229 

0.232 

0.227 

0.243 

0.000 

0.144 

-1.194 

-1  .082 

-0.847 

-1 .594 

0.439 

-0.834 

-0.813 

1 .0000 

0.9977 

0.0007 

0.0007 

0.9993 

0.0007 

0.9981 

0.9993 

0.9992 

STEP01 

59.8183 

0.989817 

N/A 

K/A 

M/A 

N/A 

H/A 

M/A 

K/A 

C-3 


t1.24S0  0.100397  M/A  M/A  M/A  M/A  M/A  M/A  N/A 

US  A91  0  0  0  0  0  0  0 

0.232  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

Z.SOS  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

0.0018  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 

STEP02  59.8183  M/A  0.81466A  M/A  N/A  N/A  N/A  M/A  N/A 

11.2480  N/A  0.388S70  N/A  N/A  N/A  N/A  M/A  N/A 

US  0  491  0  0  0  0  0  0 

0.232  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

2.305  0.000  0.000  0.000  0.000  0.000  0.000  O.COO  0.000 

0.0018  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 

STEP03  S9.8183  N/A  N/A  0.802444  M/A  N/A  N/A  N/A  N/A 

11.2480  N/A  N/A  0.3981S5  N/A  N/A  N/A  M/A  N/A 

US  0  0  491  0  0  0  0  0 

0.232  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

2.305  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

0.0018  1  0000  1.0000  1.0000  1.0000  1.00«0  1.0000  1.0000  1.0000 

STEP04  S9.8183  K/A  N/A  N/A  0.7SS401  N/A  H/A  N/A  N/A 

11.2480  N/A  N/A  N/A  0.429730  N/A  N/A  N/A  N/A 

US  n  0  0  491  0  0  0  0 

0.232  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

2.305  0.000  0.000  0.000  0.000  0.000  0.)00  0.000  0.000 

0.0018  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 

STEP05  59.6268  1.  1.  1.  1.  0.SSS328  0.6(16667  0.333333  0.333333 

11.3397  0.  0.  0.  0.  0.496929  0.471403  0.47140S  0.471405 

168  3  3  3  3  488  3  3  3 

0.229  0.989  0.989  0.989  0.989  0.000  0.989  0.970  0.952 

2.093  0.177  0.833  0.866  0.993  0.000  -1..449  -1.703  -0.220 

0.0018  0.9977  0.9993  0.9994  1.0000  1.0000  0.0007  0.0007  0.9979 

STEP06  59.8183  N/A  N/A  H/A  N/A  N/A  0.9'4460  N/A  N/A 

11.2480  N/A  K/A  H/A  H/A  N/A  0.279483  N/A  N/A 

165  0  0  0  0  0  491  0  0 

0.232  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

2.39S  0.000  0.000  0.000  0.000  0.000  0.000  0.000  0.000 

0.0018  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000  1.0000 

ENTRIES  ARE<  KEAN/STO  OF  2N0  VAR  FOR  CASE!)  NtSSlNO  FIRST  VAR 

NUK8ER  OF  CASES  KtSSIHO  1ST  VAR  HITH  VALUES  FOR  2ND  VAR 
PHI  CORRELATION  OF  MDFLAOS 

T/SIO  OF  DIFF  li;  2ND  VAR  8ETNEEN  CASES  HITH  8  H/0  1ST  VAR 
TITLE  VERSION  IMPUTE. TEMP  ON  PROC. IMPUTE  I71LXH01 


MISSINO 

DATA  REPORT 

42'  CHARACTERISTICS 

OF  CASES  HITH  MISSINO 

>  VALUES 

CFOR  EACH 

VARIABLE) 

MISSING 

VARIABLE 

M7XHTOTT 

S  E  C  0 
STEP01 

HO  VAR 

STEP02 

I  A  8  L  E 
STEP03 

STEP04 

STEP05 

STEP06 

STEP07 

STEP08 

STEP07 

59.8426 

1 

1  . 

1  . 

1  . 

1  . 

1  . 

.  0.753086 

11 .2880 

0. 

0. 

0. 

0. 

0. 

0. 

0.431216 

0. 

170 

5 

5 

5 

5 

5 

5 

486 

5 

0.227 

0.981 

0.981 

0.981 

0.981 

0.970 

0.981 

0.000 

0.945 

2.382 

0.229 

1  .078 

1  .121 

1.286 

2.028 

0.691 

0.000 

2.816 

0 . 0018 

0.9978 

0.0007 

0.0007 

0.0007 

0.0007 

0.9988 

1 .0000 

0.0007 

STEP08 

59.8848 

1 

0.900000 

0.900000 

0.700000 

0.500000 

1 

0.900000 

0.3950 

11  .1311 

0. 

0.300000  0 

.300000  0 

.450258  0. 

300000 

P. 

0.300000  0. 

.488853 

174 

10 

10 

10 

10 

10 

10 

10 

481 

0.243 

0.964 

0.964 

0.964 

0.964 

0  952 

0.964 

0.943 

0.000 

2.474 

0.325 

0.705 

0.787 

-0.415 

-0.357 

0.982 

1  .094 

0.000 

0. 0018 

0.9979 

0.9988 

0.9991 

0.9981 

0.9980 

0.9999 

0.0007 

1  .0000 

ENTRIES  ARE'  MEAN/STD  OF  2ND  VAR  FOR  CASES  MISSING  FIRST  VAR 


NUMBER  OF  CASES  MISSINO  1ST  VAR  HITH  VALUES  FOR  2ND  VAR 
PHI  CORRELATION  OF  MDFLAOS 

T/SIO  OF  DIFF  IN  2ND  VAR  lETHEEN  CASES  NITH  k  H'O  1ST  VAR 
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TITLE  VERSION  IMPUTE. TEMP  ON  PROC. IMPUTE  I7UXH01 
MISSING  OATA  REPORT  tS'  CORRELATIONS  BETWEEN  REPORTED  VALUES 


FIRST 

SECOND 

1  V  A  R  I  A 

6  L  E 

STEP07 

VARIABLE 

M7XHTOTT 

STEP01 

STEP02  STEP03 

STEPOA 

STEP05 

STEP04 

STEP08 

M7XHTOTT  1  , 

0000 

0.1915 

0.3873 

0.4103 

0.3292 

0.2248 

0.0838 

0.1892 

0.1438 

ASA 

489 

489 

489 

489 

484 

489 

484 

480 

0.0000 

0.0003 

0.0003  0. 

0003 

0.0003 

O.OOOS 

0.0003 

0.0003 

0.0003 

STEP01 

0. 

1915 

1 .0000 

0.2127 

0.2044 

0.1783 

0.1137 

0.3314 

0.1781 

0.0828 

489 

491 

491 

491 

491 

488 

491 

484 

481 

O.OOOS 

0.0000 

O.OOOS  0. 

0003 

0.0003 

0.0003 

0.0003 

0.0003 

0.0003 

STEP02 

0. 

SS75 

0.2127 

1  .0000 

0.8823 

0.7455 

0.3742 

0.1352 

0.1777 

0.2548 

489 

491 

491 

491 

491 

488 

491 

484 

481 

O.OOOS 

0.0003 

0.0000  0. 

0000 

0.0000 

0.0003 

0.0003 

0.0003 

0.0003 

STEP03 

0. 

4105 

0.2044 

0.8823 

1.0000 

0.7534 

0.4223 

0.1409 

0.1477 

0.2452 

489 

491 

491 

491 

491 

488 

491 

484 

481 

0 . OOOS 

0.0003 

0.0000  0. 

0000 

0.0000 

0.0003 

0.0003 

0.0003 

0.0003 

STEP04 

0. 

S292 

0.1783 

0.7455 

0.7534 

1 .0000 

0.4444 

0.1989 

0.1922 

0.3491 

489 

491 

491 

491 

491 

488 

491 

484 

481 

0. OOOS 

0.0003 

0.0000  0. 

0000 

0.0000 

0.0003 

0.0003 

0.0003 

0.0003 

STEP05 

0. 

226B 

0.1137 

0.3742 

0.4223 

0.4444 

1.0000 

0.1155 

0.1452 

0.4272 

484 

488 

488 

488 

488 

488 

488 

483 

478 

0. OOOS 

0.0003 

0.0003  0. 

0003 

0.0003 

0.0000 

0.0003 

0.0003 

0.0003 

STEP06 

0. 

08S8 

0.3314 

0.1352 

0.1409 

0.1989 

0.11SS 

1 .0000 

0.4492 

0.1897 

489 

491 

491 

491 

491 

488 

491 

484 

481 

O.OOOS 

0.0003 

0.0005  0. 

0003 

O.OOOS 

O.OOOS 

0.0000 

0.9003 

0.0003 

STEP07 

0. 

1892 

0.1781 

0.1777 

0.1477 

0.1922 

0.1452 

0.4492 

1  .0000 

0.2113 

484 

484 

484 

484 

484 

483 

484 

484 

474 

0. OOOS 

0.0003 

0.0003  0. 

OCt/C 

O.OOOS 

0.0003 

0.S003 

0.0000 

0.0003 

STEPOB 

0. 

14S8 

0.0828 

0.2548 

0.2451. 

0.3491 

0.4272 

0.1897 

0.2115 

1 .0000 

480 

481 

481 

481 

381 

478 

481 

474 

481 

O.OOOS 

0.0002 

0.0003  0. 

0003 

O.OW.3 

0.0003 

0.0003 

0.0003 

0.0000 

ENTRIES 

ARE'  CORR/H/SIO 

NOTE' 

472.  CASES  HAVE  COMPLETE 

DATA 

NO  MORE 

310N1F  VALUES  FOR  OBJF 

TITLE  VERSION  IMPUTE. 

TEMP  OH  PROC. IMPUTE  I71LXHD1 

MISSINO 

OATA  REPORT 

(4'  REGRESSION  EQUATIONS  FOR  EACH  VARIABLE 

DEPENDENT  MULT  I 

PREDICTOR  STO 

RAH  1 

VARIABLE  PART 

VARIABLE  PART 

VARIABLE  PART 

VARIABLE 

R2 

VARIABLES  COEF  COEF  1 

1 

NAME 

COV 

NAME 

COV 

NAME 

COV 

M7XHT0TT 

0 

CONST 

1 

0 

STEP03 

0.41055 

STEP02 

0.38731 

STEP04 

0.32921 

STEP05 

0.22477 

5TEP01 

0.19152 

5TEP07 

0.18921 

1 

STEPOB 

0.14379 

STEP04 

0.08384 

STEP02 

0.15000 

1  M7XMT0TT  0.38731  0.07702  I 

1  STCP03 

0.72329 

STEP04 

0.43797 

STEP05 

0.28839 

1  CONST 

-1.1484 

STEPOB 

0.20112 

5TEP01 

0.1384A 

STEP07 

0.10445 

STEP04 

0.10274 

5TEP0S 

0.78402 

STEP02  0.85094  1.9784  I 

STEP09 

0.08378 

STEP04 

0.07337 

STEPOB 

0.05499 

M7XMT0TT  0.08097  0.00579  1 

STEP04 

0.01902 

5TCP01 

0.00794 

5TEP07 

0.00114 

CONST 

-.94353  I 

STEPOA 

0.54809 

STEP03  0.74358  1.9822  1 

$TEP08 

0. 14844 

STEP05 

0.12720 

»,  EP02 

0.10015 

H7XHT0TT  0.02394  0.00201  I 

5TCPI14 

0.09210 

STEPOT 

0.04297 

M  '0> 

3.02174 

CONST 

-.70580  1 

STEP05 

0.21449 

STEP04  0.29714  1 .1844 

STEP08 

0.47088 

5TEP07 

0.05477 

M7*'HT!3TT  0.04749 

STEP03  0.19842  0.85405 

STEPOB 

0.02844 

STEP02 

-.02429 

5TEP01 

C. 02013 

1  CONST 

-.57958 

C-5 


STEPOS  0.39339 

STEPOS 

CONST 

0.02721 

1  . 

0 

STEPO? 

STEP02 

STEPOS 

0.12090 

0.02009 

0.00031 

STEPOO 

STEPOl 

0.11729 

0.01130 

STEPOA 

M7XMT0TT 

0.06195 
0.001 56 

STEPO«  0.OS«O1 

STEP09 

0.13100 

1 .1002 

STEP07 

0.91121 

STEPOl 

0.29330 

STEPO! 

-.03788 

STEPOS 

CONST 

0.13099 

0.99090 

-.27103 

STEPOZ 

-.01357 

M7XHT0TT 

0.01991 

STEPOS 

-.00929 

STEP07  0.29277 

STEPOO 

0.9S033 

2.0301 

STEPOO 

0.10321 

STEPOS 

O.OSIZ2 

STEPO: 

0.05759 

1  M7XHT0TT 
CONST 

0.13099 

0.01990 

-2.3207 

STEPOA 

0.03173 

STEPOS 

0.B9194 

STEPOl 

-.00220 

STEP01  0.13697 

STEPOO 

0.31702 

2.9303 

3TEP02 

0.10302 

STEPOS 

0.09190 

STEPOA 

0.06087 

H7XHT0TT 

CONST 

0.10907 

0.02021 

-2  0010 

STEPOS 

0.03901 

STEPO? 

-.00220 

STEPOO 

-.00117 

M7XHTOTTO.  19205 

STEPOS 

STEP07 

STEPOl 

CONST 

0.37237 

0.10900 

0,09303 

9.9201 

1 .2023 
9.9972 
-3.9100 

STEPOO 

STEPOO 

-.05100 

0.01307 

STEPOS 

STEPOO 

0.00203 

0. OIOSS 

STEPO? 

0.01872 

STEP02  0.00293 

STEPOA 

STEPOS 

W7XHT0TT 

CONST 

0.23100 

0.09799 

0.02902 

0.90190 

1  .3093 
0.00179 
-.71920 

( 

STEPOS 

STEPOT 

1 

-.02790 

0.01 132 

STEPOl 

STEPOO 

0.02900 

-.01121 

STEPOB 

-.01262 

STEP09  0.03923  j 

Tine  vessioH  iJ 

niSSIHO  DATA  REI 

STEP02 
STEPOS 
STEPOO 
M7XHT0TT 
CONST 
'VTE.TEMP 
»0N7  09  > 

0.93003  1.1399  | 

0.32973  0.01920  1 

0.19909  0.30790 

0.00991  0.00033 

-.72310 

ON  PROC. IMPUTE  1711 
REGRESSION  E9UAT101 

STEPOO  0.00502 

1  STEPOl  0.00391 

.XHDt 

IS  PUR  VARIABLE 

STEPOS 

0.09933 

8TBP07 

0.02727 

OEPENOCNT  HUtT 

PREDICTOR  STD 

RAM 

VARIABLE 

PART 

VARIAOLE 

PART 

VARIABLE 

PART 

variable  R2 

variables  COEF 

COEF 

NAME 

COV 

NAME 

COV 

NAME 

COV 

STEPOS  0.90099 

1 

STEPOO 

STEPOS 

STEPOA 

CONST 

0.33022 

0.19372 

0.ni9B 

1  .2012 
0.37292 
0.30379 
-.19909  1 

STEPOO 

STEP02 

1 

-.03007 

-.01990 

M7XMT0TT 

STEPOl 

0.03299 

0.00931 

STEPOr 

-.02260 

STEPOS  0.90021 

STEPOS 

0.00933 

1 

1  .3330  1 

STEPOO 

0.00157 

STEPOO 

0.03327 

M7XNT0TT 

-.01767 

STEP07 

CONST 

0.12303 

0.33302  1 
-.12130 

STEPOS 

-.01279 

STEPOl 

-.OOI3B 

STEPO? 

0.00369 

STEPOO  0.29203 

STEPOl 

STEPOS 

STEPO; 

STEPOA 

CONST 

0.29999 

0.00329 

0.90117 

0.03329 

3.0000 

0.  19720 

1 .3735 
0.19022 
-3,027^ 

M7XHT0TT 

STEPOS 

-.00030 

-.03399 

STEPO? 

-.09729 

STEPOS 

-.03622 

3TEP07  0.25901 

title  VERSIOH  IMI 
niSSlNO  DATA  REl 

STEPOO 
HTXNTOfT 
STEPCO 
CONST 
PL/TE.TEMP 
PORT  I3> 

0.93707  2.9090 

0.IS09I  0.01721 

0.10092  0.33909 

-2.3703 

ON  PROC. IMPUTE  1711 
CONDITIONAL  OISTRIl 

STEPOE 

STEPOS 

LXNOI 

OUT  IONS 

0.03703 

-.00907 

STEPOO 

STEPOl 

t.  02220 
-.00213 

STEPOS 

0.02106 

C-6 


EOR.  H  HITH  ReOR  TARGET  TARGET  CUHULATIVE  CELL  MEANS  AND  RRORORTIONS 


VALUE  DATA  MEAN 

MEAN  3. 

0. 

1 

2  3 

4 

3 

TARGET  VARIAILEi 

M7XNTOTT 

1 

433  0.000 

4.297 

0.099 

0.7  0.02 

0.4  0.01  0.4  0. 

34 

0.3 

0.74  0. 

TOTAL  MEAN*  4 

.2971  SO* 

0.099/ 

N/I  CELL 

SO*  0.199/ 

R 

SO* 

0.000 

TARGET  VARIAILE' 

3TEP02 

1 

10  0.441 

1  .200 

0.400 

0.0  0.00 

0.0 

2 

41  1.503 

1  .341 

1.494 

B.O  0.44 

0.0 

3 

134  2.374 

1  .721 

0.449 

0.0  0.20 

0.0 

4 

103  3.470 

1  .001 

0.324 

0.0  0.12 

0.0 

3 

110  4.343 

1  .733 

0.200 

0.0  0.03 

0.0 

t 

9  3.211 

2.000 

0.000 

0.3  0.00 

0.0 

TOTAL  MEAN*  1 

.3131  SO* 

0.309/ 

M/I  CELL 

SO*  0.399/ 

R 

so* 

0.147 

TARCE1  VARIAILE' 

STEP03 

1 

91  -0.471 

1  .044 

0.240 

0.0  0.93 

0.0 

2 

400  ‘.300 

1  .970 

0.171 

00  0.05 

0.0 

TOTAL  MEAN*  1 

iOZi  SO* 

0.390/ 

M/1  CELL 

SO*  0.117/ 

R 

so* 

0.774 

TARGET  VARIABLE' 

3TEP04 

1 

97  -0.4  1 

1.103 

0.304 

0.0  0.90 

0.0 

2 

394  1.397 

1.914 

0.277 

0.0  0.00 

0.0 

TOTAL  MEAN*  1 

.794i  SO* 

0.430/ 

M/1  CELL 

SO*  0.213/ 

R 

so* 

0.341 

TARGET  VARIAILE' 

ST'POS 

1 

130  -0.*71 

1.177 

0.302 

0,0  0.02 

0.0 

2 

330  1.442 

1  .493 

0.441 

0.0  0.31 

0.0 

TOTAL  MEAN*  1  .333i  SO* 

0.497/ 

H/1  CELL 

SO*  0.442/ 

R 

so* 

0.21 1 

TARGET  VARIAILE' 

STEPOO 

t 

213  0.009 

1 .034 

0.230 

0.0  0.94 

0.0 

2 

244  1,000 

1.449 

0.471 

0.0  0.33 

0.0 

TOTAL  MEAN*  1 

.3931  SO* 

0.409/ 

H/I  CELL 

SO*  0.312/ 

R 

so* 

0,319 

TARGET  VARIABLE' 

STEPOO 

1 

303  0.413 

1  .072 

0.3.14 

0.0  0.13 

0.0 

2 

104  1.144 

1.914 

0.124 

0.0  Q.02 

0.0 

TOTAL  MEAN*  1 

.9141  SO* 

0.200/ 

N/1  CELL 

so*  0.274/ 

R 

so* 

0.03B 

TARGET  VARIABLE' 

STEP07 

1 

111  -0.037 

1.439 

0.490 

0.0  0.34 

0.0 

2 

373  1.307 

1  .140 

0.347 

0.0  0.14 

0.0 

total  NEAN*  1.7331  SO* 

0.431  / 

M/I  CELL 

SO*  0.401/ 

R 

so* 

0.137 

TARGET  VARIABLE' 

STCP01 

1 

134  0,193 

1.941 

0.174 

0.0  0.03 

0.0 

2 

333  1.370 

2.0OO 

0.000 

0.3  0.00 

0 . 0 

total  mean*  1 

.9901  SO* 

0.100/ 

H/I  CELL 

SO*  0.099/ 

R 

so* 

0.022 

TITll 

:  VERSION  IMPUTE. TEMP  ON  PROC. 

IMPUTE 

I7UXMD1 

MISSZNO  DATA  REPORT  < 

13'  CONDITIONAL  DISTRIBUTIONS 

EOR. 

N  HITH  REGR  TARGET  TARGET  CUMULATIVE  CELL 

MEANS  AND  PROPORTIONS 

VALUE  data  MEAN 

MEAN  S. 

.0 

1 

2  3 

4 

3 

TARGET  VARIABLE' 

M7XNTOTT 

1 

147  -11.109 

3.331 

0,720 

0.7  0.05 

0.5  0. to  0.3  0 

.73 

0.3 

O.tl  0. 

2 

0  1.300 

0.000 

0.000 

0.3  0.00 

0.3  0.00  0.3  0. 

.00 

0.3 

0.00  0. 

3 

1  2.311 

4.203 

0.000 

0.5  0.00 

0.3  0.00  0.3  0. 

.00 

0.3 

1.00  0. 

4 

11  3.009 

4.230 

0.901 

1.0  0.01 

0.4  0.12  0.3  0. 

33 

0.3 

0.71  0. 

3 

301  4.211 

4  494 

0.792 

0.7  0.00 

0.7  0.03  0.4  0 

.21 

0.3 

0.49  0. 

4 

94  34.420 

3.073 

0.430 

0.3  o.or 

0.3  0.00  0.3  0. 

.00 

0.4 

0.44  0. 

TOTAL  MEAN*  4 

.2971  SO* 

0.902/ 

H/I  CELL 

SO*  0.747/ 

R 

SO¬ 

0.314 

TARGET  VARIABLE' 

STEP02 

1 

130  -0.190 

1.300 

0.4SI 

0.0  0.70 

0.0 

2 

341  1.431 

2.000 

0  000 

0.3  0.00 

0.0 

TOTAL  MEAN*  1 

.013/  SO* 

0.309/ 

M/1  CEIL 

SO*  0.234/ 

R 

SO* 

0.432 

TARGET  VARIABLE' 

STEP04 

1 

103  -0.404 

1  .097 

0.294 

0.0  0.90 

0.0 

A 

0.9A  0.} 


A 

3  I .00  0.3 
3  0.00  0.3 
3  1.00  0.3 
3  0.91  0.2 
A  0.91  0.2 
A  0.97  0.4 
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2 

3<8  1.394  1.930 

n.2S4 

0.0  0.07  0.0 

TOTAL  MEAN-  I.79<|  SD- 

TARGET  VARIAILE'  STEP09 

0.430; 

H/1  CELL  SO* 

0.294; 

ft 

SO* 

0.923 

« 

294  9.410  1.299 

0.458 

0.0  0.70  0.0 

1V4  1,193  1.943 

0.231 

0.0  0.09  0.0 

TOTAL  MEAN*  1.S99<  SO* 

TAROET  VARIAILE'  STEPOl 

0.497; 

M/I  CELL  SD* 

0.314; 

R 

SO* 

0.402 

1 

215  0.111  1.093 

0.247 

0.0  0.93  0.0 

2 

299  1.730  1.992 

0.473 

0.0  0.34  0.0 

TOTAL  MEAN*  1.393(  30* 

TAROET  VARIAILE'  9TEP09 

0.489; 

H/l  CELL  SO* 

0.389; 

R 

SQ* 

0.398 

1 

123  -0.333  1.991 

0.492 

0.0  0.31  0.0 

2 

398  1.489  1.989 

0.104 

0.0  0.01  0.0 

TOTAL  MEAN*  1.914i  SO* 

TAROET  VARIAILE'  STEPO ' 

0.280; 

H/l  CELL  30* 

0.248; 

R 

SQ* 

0.21  3 

1 

139  0,132  1.323 

0.499 

0.0  0.47  0.0 

2 

347  1.341  1.844 

0.392 

0.0  0.19  0.0 

TOTAL  MEAN*  1 .793l  30* 

0.431  i 

H/t  CELL  SO* 

0.409; 

R 

SQ* 

0.112 

TITLE  VERSION  ZNEUTE.TEMP  ON  PROC.  IMPUTE  I7UXMDt 

VARIAILE  mean  OP  MEAN  SQUARED  ERROR 

IMPUTED  VALUES  DUE  TO  HISSXNO  DATA 


H7XHT0TT 

38.932 

2.9492 

STEPOl 

0.983 

0.0039 

STEP02 

0.840 

0.0145 

STEP03 

0.811 

0.0094 

STEP14 

0.740 

O.OIIS 

3TEP03 

0.334 

0.0398 

STEP09 

0.891 

0.0180 

STEP07 

0.793 

Q.OSCt 

STEP08 

0.429 

0.0432 

NORMAL  EXIT  PROM  PROC  IMPUTE 
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Appendix  0 
LVI  HANDS-ON  TESTS: 

PRE- IMPUTATION  INTERCORREUTIONS  AND  RESIDUALS,  AND 
PRE- IMPUTATION  AND  RESIDUAL  MEANS  AND  VARIANCES 
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Hjtrix  of  Prc-  tat  ion  Interccrrutat  ions  aiut  Residuals  for  HUS  UR  on  the  I'andsOn  Tasks 
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es  9^l 


Hjtrix  of  Pro- li;fiutat  ioo  Inicrcorrelat ions  aiid  Residoals  for  H(>S  13fl  on  the  Hands-On  lasts 


-iiVutJi  iaii  Idtercorrelat  .jix)  Hcsiifc.'als  for  HOS  I9(  on  the  Handi-(k>  Tasks 


Hilri*  of  f*re  laputat  ion  f ntn-corrrljt ions  <nd  Residue!;,  for  NDS  19K  on  the  Hents-On  Tasks 


—  r^O> 
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O  O  CQ  \0  ^  ^  mm  ^  Q  mm 

mm  t\f  mm  CSt  tO  CM  I 

—  mm  C\, 


COVCM»/>r*^RrCOcnc^cnO'^r^ 

(•n*— •i/le-«.s/>F^Q»ro^r*noQO'^ 


ooooooooooooo 

•  f  1 
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v/>  ^  CM  kC  rP  «  CO  ^  O 
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CM  0>  f**)  CM  V  CM  M  Cm  .’*» 


c^k/^cn^cnkOko»— ooocmu^co 


ui  o  on  CO 

CO  CO  CD  C^* 


^  O  kO  O  V  re^ 

CTI  o^  t/1  <“  ® 


•— tfMco^u^»0'^GOO>0  —  ' 
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Table  E-2 

Pre-impuUttion  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  M^]rS  11 B 
Prc-lmputed  Residual 
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Pre-imputatioo  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  13B 


Prc-Imputed 
Mean  Variance 


Residual 

Mear  Variance 
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0.48 

0.00 

0.00 
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Table  E-6 


Pre>itnputation  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  19E 


Prc-lmputed 

Mean  Variance 

Mean 

Residual 

Variance 
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4.73 

0.45 
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Table  E-8 


Prt-imputatlon  and  Residual  Means  and  Variances  on  all  Criterion  Measures  Tor  MOS  19K 


Prc-Imputed  Residual 

Mean  Variance  Mean  Variance 


1 

4.74 

0.55 

0.00 

0.01 
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3.83 

12.58 
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0.21 
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0.36 
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Table  E-10 


Pre>iaiputat1on  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  M05  31C 


Pre 

Mean 

-Imputed 

Variance 

Mean 

Residual 

Variance 

1 

4.72 

0.79 

0.02 

-0.03 

2 

2.97 

8.88 

-0.04 

-0.22 

3 

038 

0.70 

0.00 

0.00 

4 

236.96 

997.84 

0.05 

-47.76 

5 

2.24 

0.55 

0.01 

-0.01 

6 

50.00 

22.91 

0.10 

0.28 

7 

4.67 

1.00 

0.02 

-0  04 

8 

4.43 

0.93 

0.01 

-0.03 

9 

4.67 

1 10 

0.01 

-0.01 

10 

4.79 

1.05 

0.00 

-0.02 

11 

17.61 

41.45 

0.03 

-0.71 

12 
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26.93 

0.23 

0.88 

13 

20.69 

25.39 

0.00 

0.51 

14 

74.10 

219.26 

-0.44 

7.96 

15 

11.62 

3.06 

0.01 

0.06 

16 

20.96 

23.03 

0.04 

-0.38 

17 

9.95 

9.81 

-0.01 

0.04 

18 

6.47 

5.27 

0.00 

0.03 

19 

22.S4 

24.05 

0.07 

-0.39 

20 

53.79 

171.36 

0.00 

1.08 

21 

3.80 

2.26 

0.00 

-0.02 
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Table  E-12 


Pre-iniputation  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  63B 


1 

Mean 

Pre-lmputed 

Variance 

Mean 

Residual 

Variance 

1 

4.53 

0.79 

0.00 

-0.02 

2 

2.63 

7.65 

0.01 

•0.02 

3 

036 

0.76 

0.00 

0.01 

4 

23531 

1139.60 

031 

D.98 
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2.17 

036 
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0.00 
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49.99 

23.95 
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0,17 
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4.41 
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4.18 
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0.00 
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9 

4.58 

1.09 
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-0.01 

10 

4,66 

1.05 

-0.01 

•0.01 

11 

31.94 

3134 

0.07 

-0.03 

12 

21.51 

19.30 

0.04 

-0.25 

13  ■ 

8237 

99.15 

0.37 

-0.56 

14 

940 

9.57 

-0.07 

022 

15 

21.93 

22.05 

-0.05 

0.37 

16 

18.62 

12.98 

0.01 

0.08 

17 

57.46 

155.31 

0.01 

1.06 

18 

15.64 

15.52 

-0.02 

0.27 
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Matrii  of  Prc-imputatioa  latefrMrrelotioos  aod  Residuals  on  all  C'riirrion  Measoies  for  MOS  711. 
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Prc’lmpuution  aad  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  71 L 


Pre-Impulcd 
Mean  Variance 


Residual 

Mean  Variance 
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V  I 

§o 


^  ^  <N  ^ 

8^8988 


<N 

?  9 


o  ^ 
o  8  o  ^ 


8 


8?? 

WO  o  Q  •; 

,.«x  W  ^  ^ 


•••  Q 
O  ^ 


888 


^  *0  fO 

3  c  8  S 


3? 

i  ■  i 


s  o  i  8  8 


S  5 


2  5*  g 


04 


f  4 
8 


S 

o 


—  a 
^  c 


—  xa^xr  - 

C  2  O  5>  8  8 


04  —a 

04  o 

O  2  o 


»  Ov  Q 

£a~«  »0 

sO  O 


o 

o 


s 

9 


-o 

8 


T'  wo  OJ  fO  AO 

•.  rv/  ^  01 

90000 
5  3  ?3  £  * 

00090 


O  30 

o  o 


oj  — 

5  2c 


w/^  <N  — • 

0090 


i!?§S 


2  s  2  g  a' 
00282 


J  30  —  U* 

o  o  2  2  2 


w/1  « 

S  S 


^  rsi  •  o 
88  2 


N©  '  A- 

a  2 


lA  ^ 

v-»  ao  - 


00  00 

8  2  2 


r-s  00 

Swri  ^ 
<NI 


Wf->  On 
r4  AO  v^ 
O 


• 

w 

0 

0 

0 

p 

jt 

Owl 

r* 

w-l 

0 

9 

9 

•r 

04 

c^ 

wn 

5 

2 


-8^2 

o  o  2  5 


c 

30 


o 

o 


s 


8 


n£)  <N  2  '  C  'O 

—  o  2  ri 
coo  'n 


§§§ 


as 

o  o  o 


I  2  ^ 
i  s  2 


^*>3  —  —  ^ 


rM'C<^^^0‘0‘“0 

Sw^  'O  aO  «  H  W 

o  —  —  o  ^  o  o 


*f 

Of 

o 


8 


r*»  '  ^  rs  r^  r^ 

8  s  S  t  2 


—  Wf  go  n*  ^ 

—  0^1/^  rJ 

O  O  — 


'T  fO  1/^  Q  00 

rM  04  n"  C  V 


'/^  n  A-  00  C  '"I 
o  0  —  N  f-*  00 


—  ^A^r•lc^fn  —  onOoo 
p  a  ^  ’n 

O  '.^0009000 


ror*-<M^f-^o---o4r-« 

<N  (^  Q  -f  o  o 

‘^'-OOOOCOO 


^rr^r>iQoo—  —  r^fO 

Of'^.  rngg^ogoooj 


—  r^ro^iO'0r^00C^'C^r*Jr^-Tt4^vCf^ 


£-17 


1 

o 

tL 

t9 

C. 


c 

o 


3 

“O 

•a 

u 

a: 


s 

(0 

*<3 

c 

3) 

<0 


c 

D 


<9 

s 

C 

o 

'«rf 

44 

ha 

o 

u 

w 

44 


e 

•2  -o 

i2  - 

5  .‘ii 
^  S 
c  ^ 
"T  /> 

-  1 
CL  8 

sS 

h  o 


44 

Z 


Table  E-16 


Pic-iniputation  and  Kciidual  Means  and  Variaoces  on  all  Criterion  Measures  for  MOS  88M 


M  / 

F  iv.-Iraputed 
j  Variance 

Mean 

Residual 

Variance 
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0J6 

0.02 
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3.o5 
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0.44 
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0.00 

4 

2:0.82 

1.36 1.41 

-0.15 

-40.13 

■z 

2.02 

0.54 

0.00 

0,|\» 

t' 

50.00 

25.21 

-0.02 

0.0- 

7 

4.49 

0.91 

0.00 

0.00 

8 

4.29 

0.82 

0.01 

-0.(j2 

9 

4.52 

0.95 

0.01 

0  DO 

10 

4.69 

0.93 

0.01 

0.00 

11 

40.18 

115.58 

0.23 

0.76 

\z 

75..’.7 

94.23 

0.16 

-2.14 

13 

32.07 

41.92 

0.10 

1.09 

14 

25.19 

33.27 

0.03 

0.04 

15 

:.ii 

1.75 

0.01 

0.01 

16 

34.09 

33.05 

0.00 

-0.05 

17 

33.27 

49.17 

0.00 

-0.32 

E-18 


Miitrix  of  Pi¥-iniputution  Iniercorrelations  and  Kcsidiiais  on  all  Crilerion  Measure;  for  MOS  9IA 
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Table  E-18 


Pre-impuiatiua  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  91A 


Pre-Imputcd 

Mean  Vaunance 

Residual 

Mean  Var.aicc 

1 

4.73 

0.63 

0.00 

-0.03 

3.57 

10  56 
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Matrix  of  !*i^-iiiipu(uti<>n  ItiltrcorreluUon!.  and  Residuals  on  all  Crilerion  Measures  for  MOS  9511 
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Table  E-20 


Pre-imputation  and  Residual  Means  and  Variances  on  all  Criterion  Measures  for  MOS  95B 
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Appendix  F 

LVI  JOB  PiHFOkMANCE  MEASURE  SUMMARY  STATISTICS 
FOR  10  MOS 
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